Lo-Fi Python

The Problem and Solution

How should the DBMS represent the database in storage files on disk? Solve it by choosing the right storage model for your target workload. The right strategy varies if you are reading data, writing data and with how many joins you are performing.

Workload Characterization

OLTP (Online Transaction Processing): "Simple queries with lots of writes."

OLAP (Online Analytical Processing): "Read only queries. Lots of joins. Doing a lot of reads, but they're more complex."

HTAP (Hybrid Transactional Analytical Processing): "is trying to do both of them. You still want to ingest new data, but analyze it as it comes in. It's used for companies making decisions on the fly as people are browsing websites, like internet advertising companies."

Storage Models

screenshots from the lecture

N-ary used to be the dominant model until the '80s.

Additional Reading: All Things Distributed

Column Store Vs. Row Store RDBMS

Row-oriented DBMS(Row Store)

• PostgreSQL, MySQL
• Row Store = use OLTP

Column-oriented DBMS(Column Store)

• Red Shift, BigQuery
• Column Store = use OLAP

If types are consistent, you can compress data into single column store.

University CS Courses For Free

CS50's Web Programming with Python and JavaScript | Harvard University

CS 61-C Great Ideas in Computer Architecture (Machine Structures), Spring 2015 | UC Berkeley

CS 109: Data Science, 2015 | Harvard University

Mathematical Modeling of Football, Fall 2020 | Uppsala Universitet

CS 162 - Operating Systems and Systems Programming, Fall 2013 | UC Berkeley

15-445/645 Intro to Database Systems, Fall 2019 | Carnegie Mellon University

15-721 Advanced Database Systems, Spring 2020 | Carnegie Mellon University

Missing Semester: Shell Tools & Scripting, Spring 2020 | MIT

6.824 Distributed Systems, Spring 2020 | MIT

CSE 373 - Analysis of Algorithms, 2016 | Stony Brook University

CS 4150 Algorithms, Spring 2020 | University of Utah

CS 241 System Programming, Spring 2020 [course wiki] | University of Illinois

CS 6120: Advanced Compilers: The Self-Guided Online Course | Cornell University

Intriguing Coursera Classes

DevOps Culture and Mindset | UC-Davis

Computer Science: Algorithms, Theory, and Machines | Princeton University

Excel Fundamentals for Data Analysis | Macquarie University

Build a Data Science Web App with Streamlit and Python | Guided Project [$10]

Programming Talks & Tutorials

These programming talks piqued my interest, highly recommended.

David Beazley | Built in Super Heroes [YouTube]

Mr. Beazley shows how to use pure Python built-in functions to clean and analyze the City of Chicago's food inspection data. No pandas in this talk, behold the power of the Python standard library. Spoiler: Don't eat at O'hare airport. He also has a new course, available for free:

David Beazley | Practical Python Programming [Course]

This is not a course for absolute beginners on how to program a computer. It is assumed that you already have programming experience in some other programming language or Python itself.

Sebastian Witowski | Modern Python Developer's Toolkit [YouTube]

An overview covering editing tools and setup from PyCon 2020. Honing your development environment is crucial to being an efficent coder. This example uses VS Code. I use Atom as my primary text editor. The most recommended linters are usually pylint, flake8 or pyflakes.

Jake VanderPlas | Reproducible Data Analysis in Jupyter [YouTube]

This 10 video series is a must-watch for aspiring data scientists and analysts if you use Python. Includes a git workflow demonstration, working in Jupyter Notebooks and many other essentials.

Rich Hickey | Hammock Driven Development [YouTube]

Sometimes, the best thing we can do is step away from the keyboard. I really enjoy this speaker's communication style.

Eric J. Ma | Demystifying Deep Learning for Data Scientists [YouTube]

Tutorial-style Pythonmachine learning walk-through from PyCon 2020.

Julie Michelman | Pandas, Pipelines, and Custom Transformers [YouTube]

This video shows a deep dive into the world of sci-kit learn and machine learning. PyCon and PyData videos usually include some cutting edge tech. Machine learning moves so fast there are always new tools surfacing. But certain libraries like sci-kit learn, TensorFlow, keras and PyTorch have been constant.

Ville Tuuls | A Billion Rows per Second: Metaprogramming Python for Big Data [YouTube]

Make your data dense by tactically re-arranging into efficient structures and compiling it down to lower-level bytes. This details a successful Python / Postgres / Numba / Multicorn big data implementation.

Video & Course Grab Bag

Discover the role of Python in space exploration [course]

Microsoft and NASA made a free course about Python in space! 🤓

Ted Nelson | Computers for Cynics [YouTube]

I find these videos to be an entertaining, thought-provoking take on software history. Recommended from Joe Armstrong, the creator of Erlang.

GNU Typist [Tutorial]

You may be able to teach yourself to type more efficiently with this tutorial. I definitely need to do this. It's worth mentioning, per Rich Hickey: with a proper design phase, you'll spend less time typing in the first place!

Extra Credit: Python Wikipedia Library

import wikipedia [GitHub]

Supplementary Resources

• teachyourselfcs.com
• Cool Python Tips Jupyter Notebook
• Google's Python Style Guide
• Python Cheat Sheet (Github)
• Free and Open Machine Learning
• AutoML and The Rise of the Citizen Data Scientist
• Data Engineering Study Guide
• The Art of Command Line
• Open Source University
• Data Training Materials from the New York Times
• freecodecamp.org

Essential MySQL Terminal Commands

Show MySQL Version

1

SELECT VERSION();

List tables in a database

1

SHOW TABLES;

Show All MySQL Variable Values

1

SHOW VARIABLES;

Creating a Table

1

CREATE TABLE Marijuana (id INT AUTO_INCREMENT PRIMARY KEY, email VARCHAR(350), date VARCHAR(350));

Create a Table with a JSON column

1
2
3
4
5
6

CREATE TABLE Marijuana (
  `id` mediumint(8) unsigned NOT NULL AUTO_INCREMENT,
  `store` varchar(200) NOT NULL,
  `details` json DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB;

Add New Column and specify column to insert AFTER

1
2

ALTER TABLE Marijuana
ADD COLUMN date VARCHAR(100) AFTER other_column;

Alter Datatype of a Column

1

ALTER TABLE Marijuana MODIFY id INT AUTO_INCREMENT PRIMARY KEY;

Describe a Table

1

DESCRIBE Marijuana;

View All Records in a Table

1

SELECT * from Marijuana;

Using LIKE in MySQL

Select 10 Newest Records

1

SELECT * from Marijuana ORDER BY id DESC LIMIT 10;

"Explaining" A Query

1

EXPLAIN SELECT * from Marijuana;

Using "ANALYZE TABLE" tooptimize themis periodically recommended by MySQL:

1

ANALYZE TABLE Marijuana;

Installing Libraries in PythonAnywhere

You can use pip to install python libraries within the PythonAnywhere bash terminal. Go to the consoles tab and start a new bash terminal. Then to install a library, such as pandas:

python -m pip3.8 install --user pandas

Flask app with mysql.connector API, SQL and pandas

A Flask app making a mysql database connection with pandas:

1. Creating an error log with logging.
2. Connecting to a mysql database hosted through Flask and pythonanywhere
3. Then reading a table to a pandas dataframe

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84

import mysql.connector
from flask import Flask
import pandas as pd
from datetime import date
import logging
import sys

app = Flask(__name__)
logging.basicConfig(stream=sys.stderr, level=logging.DEBUG)


@app.route("/")
def hello_world():
    """Call database and return data from df. Then display homepage."""
    try:
        email_df = get_database_table_as_dataframe()
        print(email_df.shape)
        html_page = render_homepage()
        return html_page
    except:
        logging.exception("Failed to connect to database.")


def render_homepage():
    """Note: you should use Flask's render_template to render HTML files.
    However, for example you can make a quick f-string HTML page that works in this code.
    """
    html_page = f"""<html><head><link rel='stylesheet' href="/static/styles/some_file.css"><link rel="shortcut icon" type="image/x-icon" href="static/favicon.ico">
                    <Title>Dispensary Alerts</Title></head>
                    <body><h2></h2>
                    <p>Get alerts for your dope.</p><br>
                    <h6><b>Sign Up</b></h6><br>
                    <div class="form">
                    <form action="/add_signup_to_db" method="post" style="width:420px;text-align:center;display:block;" >
                    <input type="text" name="Signup Form">
                    <input type="submit" value="Submit">
                    </form></div><br><br>
                    <p><b>Current Time:</b>
                    {str(date.today())} </p></body></html>"""
    return html_page


def get_database_table_as_dataframe():
    """Connect to a table named 'Emails'. Returns pandas dataframe."""
    try:
        connection = mysql.connector.connect(
            host="username.mysql.pythonanywhere-services.com",
            db="username$DatabaseName",
            user="username",
            password="password",
        )

        email_df = pd.read_sql(sql="""SELECT * FROM Emails""", con=connection)
        logging.info(email_df.head())
        return email_df
    except:
        logging.exception("Failed to fetch dataframe from DB.")
        return "Oops!"


@app.route("/add_signup_to_db", methods=["GET", "POST"])
def add_signup_to_db(email, date):
    """Pass data as SQL parameters with mysql."""
    try:
        connection = mysql.connector.connect(
            host="username.mysql.pythonanywhere-services.com",
            db="username$DatabaseName",
            user="username",
            password="password",
        )
        cursor = connection.cursor()
        sql = """INSERT INTO Emails (message, date) VALUES (%s, %s) """
        record_tuple = (email, date)
        cursor.execute(sql, record_tuple)
        connection.commit()
    except mysql.connector.Error as error:
        logging.info("Failed to insert into MySQL table {}".format(error))
    except:
        logging.exception("Error inserting records to DB.")
    finally:
        if connection.is_connected():
            cursor.close()
            connection.close()
        return "MySQL connection is closed"

Iterative Development

Below: making my website look less like a "my first HTML" website, experimenting with my app's message name and adding a sign-up form connected to the database.

Note: if you see this error when making a request in pythonanywhere:

OSError: Tunnel connection failed: 403 Forbidden

It's likely because you are "whitelisted" on the free plan. Upgrading to the $5/month plan will fix it!

Scoping The Full Stack

I'm really enjoying this web development stack. Here are all of the tools and library choices for this website:

• HTML
• CSS
• web framework: Flask library
• email: Flask-Mail library(SMTP)
• API calls to external websites: requests and json libraries
• data handling: MySQL database, mysql.connector API, pandas library
• file system: logging, os and sys libraries
• (may add) payment processing: Braintree Library
• web hosting: pythonanywhere

Finding Your Flask Groove

Flask is a little scary at first, but reasonable once you get a grasp of the basic syntax. Using the logging module to establish access, error and server log feeds was a big step to finding my Python traceback fixing groove. It's a work in progress.

Recapping My Python Web Development and Database Experiences

I previously created a website with web2py, another Python web framework like Flask and Django. I think it was a decent choice for me at that point in my Python journey. Back then, I connected a MongoDB back-end to web2py. I randomly picked Mongo out of the DB hat and it worked well enough.

My Python Web Development and Database Tools

App #1 web2py + MongoDB

App #2 Flask + MySQL

Future App? py4web + pyDAL + PostgreSQL

Future App? tornado + streamlit (or) Flask + Dash (+ SQLite)

Of these two diverse Python stacks, I favor MySQL and Flask. But I learned a lot from watching web2py's tutorial videos and it's less intimidating for beginners. And I barely scratched the surface of web2py's "pure Python" pyDAL (Database Abstraction Layer), which seems pretty dope.

web2py's creator has a new framework in progress called py4web. It has the same DAL and inherits many other web2py qualities. Definitely looking forward to exploring the DAL on my first py4web website. I'll likely use it to connect to PostgreSQL or SQLite. Maybe I'll install pyDAL with pip in the meantime.

Final Thoughts

Both of my websites are hosted with pythonanywhere, which gives you a text editor and bash terminal to run your scripts in a shell environment. I'm so pleased with all of these tools. They fit together smoothly and made creating my website a fun experience.

API & HTTP Lingo You Should Know

Hypertext Transfer Protocol (HTTP)

Per Mozilla, "Hypertext Transfer Protocol (HTTP) is an application-layer protocol for transmitting hypermedia documents, such as HTML. It was designed for communication between web browsers and web servers, but it can also be used for other purposes. HTTP follows a classical client-server model, with a client opening a connection to make a request, then waiting until it receives a response."

HTTP: you = client. API = way to communicate with server

Application Programming Interface (API)

Per Wikipedia, the purpose of an API is to simplify "programming by abstracting the underlying implementation and only exposing objects or actions the developer needs."

Representational State Transfer (REST)

REST is an architectural style of web APIs. It is the dominant architecture that many APIs use. Simple Object Access Protocol (SOAP) is another style I've heard of, but it seems less common nowadays.

A REST API is built for interoperability and has properties like: "simplicity of a uniform interface" and "visibility of communication between components by service agents." [Wikipedia] If an API follows REST, it has many good principles baked in.

GET, POST and PATCH

These are three common types of request methods.

• GET: Read data returned, such as all of your tweets in the Twitter API.
• POST: Create a new item, like writing a new tweet. Can also update existing data. Tweets aren't editable though!
• PATCH: Similar to POST, this is typically used for updating data.

URL or "Endpoint"

This is the website target to send your request. Some APIs have multiple endpoints for different functionality.

URL Parameters

Values you pass to tell the API what you want. They are defined by the API specifications, which are usually well documented. In Python's requests library, they may be passed as keyword arguments. Sometimes they are passable directly within the endpoint url string.

Body or "Payload"

To make a request, you send a payload to the url. Often this is a JSON string with the API's URL parameters and values, AKA the request body. If the API is written specifically for Python, it might accept an actual Python dictionary.

Javascript Object Notation (JSON)

JSON is the data interchange standard for all languages. Usually it is the default way to pass data into and receive data from an API. If making a POST, you can check your json object is formatted correctly by using a json linter. Or try Python's json.tool! You can also pretty print your JSON or python dictionary with the pprint module. If you're using json.dumps remember it has pretty printing accessible by keyword arguments! These features are accessible in the standard library. Isn't Python great? See also: Python 101 - An Intro to Working with JSON

Pages

API data is commonly returned in multiple pages when there is a lot of data returned. Each page can be accessed one request at a time. Sometimes you can specify how many items you want on a page. But there is usually a maximum items per page limit like 100.

Status Code

Each request usually gives you a numeric code corresponding to happened when the server tried to handle your request. There is also usually a message returned.

Headers

These usually contain website cookies and authorization info. They also may tell the API what kind of data you want back. JSON and XML are the two most common types of data to return. You can specify the return format in the content-type headers.

If you need to parse an XML response, check out Python's stock ElementTree API. I've only seen a few APIs using XML responses, such as the USPS Address Validation API.

Authorization

Authorization varies widely. This is the level of identification you need to pass to the API to make a request. Public APIs might require none. Some just need a username and password. Others use the Oauth standard, which is a system involving credentials and tokens for extra security.

Authorization Scheme Example [Mozilla]

Authorization: <auth-scheme> <authorisation-parameters>

1
2

# headers python dict example
headers = {"Authorization": f"basic {token}"}

Creating the Request JSON

I recommend using Postman in most cases, depending on the complexity of the API. If the JSON syntax is straightforward, you can format your data as a python dictionary, then convert it to a JSON object with json.dumps from the standard library's json module. But JSON can be tricky sometimes. You may also need to pass a dictionary of HTTP headers.

Some APIs have "Postman Collections", a set of Python (or any language) script templates for the API. In those cases, it might make sense to use those resources.

Path One: Make HTTP request with json & requests libraries

Format Python dict with json.dumps from the standard library's json module. Infer API requirements from documentation. Use requests for HTTP.

Path Two: Make HTTP request with Postman & requests library

Use Postman to generate the JSON payload. Plug headers and payload into requests. Use requests library for HTTP.

Postman has a friendly interface for plugging in all your pieces and tinkering with your request body until it works. Make it easier on yourself and use Postman, especially if there are collections. An alternative is to troubleshoot in Python if you are confident in your grasp of the API. I use both options depending on my familiarity with the API at hand.

Formatting Your Request

1. Once you have the request working, you may export your Postman request to almost any language. For Python, you can sometimes export to the requests, http.client or urllib libraries. Hit the "code" button in Postman and then copy your code.
2. Paste your Postman headers, payload and url into your existing code.
3. You may want to use a dict or string formatting to pass values to your request parameters or url.
4. If the API uses a token or other form of authorization that needs to be refreshed intermittently, I usually have a function that returns a token. token = fetch_token() Then put the token in the headers dict. {"Authorization": f"basic {token}"} Finally pass your headers and payload to your requests.get, requests.post, or requests.request function along with the endpoint url. You're now ready to test the request.

If you choose not to use Postman, you can use the json library. See the use of json.dumps() to convert a dictionary to a JSON object in example #2 below.

Python Installation

You can install requests with pip. Alternatively, http.client is included within the Python standard library. If you want to convert HTTP response data to a dataframe or csv, install pandas.

python -m pip install requests
python -m pip install pandas

Example #1: GET the geolocation details of any public location with the Google API

This was modified from another example of Google's Geolocation API. To use this, you need to create a developer account with Google and paste your API keys below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import requests


# Find the best double-cheeseburger + fries $7 can buy.
payload = {"key": "Add_Google_API_Key_Here", "address": "Redhot Ranch"}
url = "https://maps.googleapis.com/maps/api/geocode/json"
# Optional: set a 5 second timeout for the http request.
r = requests.get(url=url, params=payload, timeout=5)
print(r.text)
print(r.status_code)
data = r.json()

# Extract the latitude, longitude and formatted address of the first matching location.
latitude = data["results"][0]["geometry"]["location"]["lat"]
longitude = data["results"][0]["geometry"]["location"]["lng"]
formatted_address = data["results"][0]["formatted_address"]
print(longitude)
print(latitude)
print(formatted_address)

# Optional: convert response into a dataframe with pandas.
# import pandas as pd
# location_df = pd.json_normalize(data['results'])
# location_df.to_csv('Locations.csv')

Above you can see:

• requests makes it easy to see the server's text response also with response.text
• requests also makes JSON encoding easy with response.json()
• pd.json_normalize() is convenient to convert the response dictionary to a dataframe.

Example #2: Encode a Python dictionary to json string and POST to a hypothetical API

1. Create a dictionary with request body data and pretty inspect it with pprint.
2. Encode the json string with json.dumps from the standard library's json module.
3. POST the encoded JSON to the endpoint url with requests.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

import pprint
import json
import requests


def dict_to_json_data():
    """Create request body with fictional contact details."""
    payload = {
        "first_name": "P",
        "last_name": "Sherman",
        "address": "42 Wallaby Way",
        "address_2": "",
        "city": "Sydney",
        "state": "NSW",
        "country": "AU",
        "zip": "2000",
    }
    pprint.pprint(payload)
    json_str = json.dumps(payload, ensure_ascii=True)
    # Optional: encode json str to utf-8.
    return json_str.encode("utf-8")


def post_data(json_str):
    """This is a fictional API request that passes a json object to requests.
    It decodes the server response with response.json() and
    Returns dictionary value by calling the data's keys.
    """
    headers = {
        "Authorization": f"Bearer {token}",
        "Content-Type": "application/json",
        "cache-control": "no-cache",
    }
    r = requests.request(
        method="POST",
        url="https://SomeSoftwareAPI.com/people/",
        data=json_str,
        headers=headers,
    )
    data = r.json()
    print(data.keys())
    # Call dict keys to get their values.
    contact_id = data["contact_id"]
    return contact_id


json_str = dict_to_json_data()
contact_id = post_data(json_str)

requests.request keyword argument alternatives for passing data

params – (optional) Dictionary, list of tuples or bytes to send in the query string for the Request.

data – (optional) Dictionary, list of tuples, bytes, or file-like object to send in the body of the Request

json – (optional) A JSON serializable Python object to send in the body of the Request

[requests API documentation]

"Gotchyas" To Avoid

• Status codes are your friend. They offer a hint at why your request is not working. If you see 200 or 201, that's a good sign. They're usually helpful, but sometimes they can be misleading.
• Ensure you are defining the correct content-type. I had an experience where Postman defined two conflicting content-type headers and it caused my request to fail. The server's error message indicated the problem was in my JSON, so it took me a while to figure out the headers were the problem.
• Sometimes it makes a difference if your url has http:// vs. https:// in it. Usually https:// is preferred.

Sidebar: requests.Session()

You might be able to improve performance by using a requests "session" object.

1
2
3
4
5
6
7
8
9

import requests


# A session adds a "keep-alive" header to your HTTP connection + stores cookies across requests.
s = requests.Session()
for page in range(0, 2):
    url = f"https://exampleapi.com/widgets/{str(page)}"
    r = s.get(url)
    print(r.text)

Dig deeper into requests by raising your HTTPConnection.debuglevel

HTTPResponse.debuglevel: A debugging hook. If debuglevel is greater than zero, messages will be printed to stdout as the response is read and parsed. Source: http.client Python Docs

1
2
3
4
5
6
7
8
9

from http.client import HTTPConnection
import requests


HTTPConnection.debuglevel = 1
payload = {"key":"Add_Google_API_Key_Here", "address":"90 Miles"}
url = "https://maps.googleapis.com/maps/api/geocode/json"
r = requests.get(url=url, params=payload, timeout=5)
print(r.text)

"As described in PEP3333, the Python Web Server Gateway Interface (WSGI) is a way to make sure that web servers and python web applications can talk to each other." Gunicorn is one of a few Python WSGI clients. web2py is another WSGI client and web framework I have used.

Conclusion

I remember APIs seemed mysterious and daunting before I had used them. But like all things, they can be conquered with knowledge, understanding and tenacity to keep trying until you figure it out. Good luck!

Requests Documentation

requests.request() API documentation

requests.get() API documentation

requests.post() API documentation

Supplementary Reading

Google's HTTP Timing Explanation

List of Interesting "Unofficial" APIs

Proxy servers

Making 1 million requests with python-aiohttp

Nginx

Create, read, update and delete (CRUD)

Helpful Windows Command Line Documentation

• Windows command line Documentation Syntax
• Command-line reference A-Z
• Command shell overview
• Using command redirection operators
• About Windows Remote Management

A Few General Windows Commands

Use find to look in a text file to count the lines matching a string:

find /C "FAIL" < "Test_Results.txt"
# returns: 0 if no match or # of lines found, e.g. 2,50,100

I wrote a post on findstr, which offers similar functionality.

• clip: pipe commands into the clipboard.
• If: If Statements based on if files exist.
• List ip address-related info:

ipconfig

Check system bit (usually 64-bit or 32-bit):

wmic os getosarchitecture

Automate Windows Scripts with Batch Files

Batch files can be run from command prompt or by double-clicking them. Here's an example of text in a batch file that activates a python virtual environment. Swap in your username and environment if you've created it.

cmd /k "cd C:\Users\your_username\PythonEnv\Scripts & activate & cd .. & dir"

1. Save above as a .bat file.
2. This uses cmd to open a new command prompt in a Windows batch file.
3. cdinto my python virtual env then activate it by running a batch file.
4. Then call dir to print directory contents.

Set a custom system 'last_name' variable to be recalled later.

set /p last_name=Enter a last name:
echo %last_name%
pause

Here we print it out with echo. Then pause.

Line continuation in batch files: Use ^ to continue your batch file scripts on a new line.

System Assessment Tools: powercfg and sfc

Display system stats:

systeminfo

Use powercfg to assess power, sleep and system states

powercfg /SLEEPSTUDY

Use sfc to perform a system file check:

# scan and repair
sfc /SCANNOW
# scan, but do not repair:
sfc /VERIFYONLY

Accessing a Remote Computer From the Command Line

You may want to ping a remote computer to see if it's running. Add your ip address instead of the below 1s and 0s:

ping 01.10.10.01

Log into your Remote Desktop with mstsc:

1. Run Remote Desktop Connection, save an RDP file from Windows Desktop Client.
2. You may need to adjust your credentials on your local machine.
3. Finally, trigger login to an active window from command prompt:

mstsc RDP_File_Name.rdp

WinRM and WinRS can allow terminal access to your Remote Desktop. You may need to set your wifi network to private. To configure winrm:

winrm quickconfig

Log into a remote computer with winrs and run ipconfig:

winrs -r:https://myserver.com -t:600 -u:administrator -p:$%fgh7 ipconfig

Check Out Python's pywin32 Module

This module is extremely useful for scripting out Windows applications. For example, I've made good use of its interfaces to Outlook and Task Scheduler. Install with pip:

python -m pip install pywin32

Here's an example to send an Outlook email:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

import win32com.client

outlook = win32com.client.Dispatch('outlook.application')
mail = outlook.CreateItem(0)
mail.To = '[email protected]'
mail.CC = '[email protected]'
mail.Subject = 'Moneyball Review'
mail.Body = """Moneyball is an inspiring movie, based on real events.
            Brad Pitt, Jonah Hill and Philip Seymour Hoffmann gave great performances.
            The trade deadline scene is delightful. Wow.
            Chris Pratt as Hatteberg too. What a solid film.
            Money isn't everything. Playing ball is.
            """
mail.Attachments.Add('Baseball_Analysis.csv')
mail.Send()

This post covers 6 handy Windows tools:

1. Windows Experience Index
2. Disk Cleanup
3. Windows Update Troubleshooter
4. Windows Update
5. Microsoft Support and Recovery Assistant
6. Disk Defragmenter

First, get your baseline Windows Experience Index score.

These metrics are a way to monitor your system's performance over time.

1. Go to "Control PanelSystem and SecuritySystem"
2. Select "Your Windows Experience Index needs to be refreshed."
3. Note the metrics you see and the last updated date. If you've never refreshed them, these are likely your computer's factory scores.
4. Then click "Refresh Now" and note the results. These are your computer's current scores.

Run Disk Cleanup

The name says it all. Go to the Start menu and search 'Disk Cleanup'. Running this freed up 40 GB of C: drive space for me.

Install OS patches and software updates with Windows Update.

First, troubleshoot Windows Update to fix any errors.

The Windows Update Troubleshooteris supported by Windows and may help with updates that fail. I downloaded the troubleshooter for Windows Update and BITS, the Windows Background Intelligence Transfer System.

The troubleshooter analyzes Windows Update and tries to fix the errors it finds. After running, it provides a status update on the issues it finds.

Run Windows Update to upgrade your software.

Windows Update usually updates your software reliably. However, some updates may fail or are not triggered automatically. Installing updates, especially security patches for your operating system is typically a good idea.In my case, several Windows 7 OS security patches had not auto-updated, some from 6 months ago.

Checking for Software Updates

• Go to your start menu and search for 'Windows Update'.
• I clicked 'Check online for updates from Windows Update' also.
• When you restart your computer, use a power cord for your laptop.
• I found more new updates twice after installing new updates and restarting my system.  Some updates are required before another update may be installed.

Microsoft Support and Recovery Assistant

Got Microsoft Errors? Check out the Microsoft Support and Recovery Assistant. It may help you if you're having trouble with Microsoft Office, Skype or any other Windows tools.

Finally, defragment your C: drive.

Defragmentation is like spring cleaning for your computer's hard disk. It optimizes your drive's data for more efficient computing and frees up space for other activities.

1. Go to your start menu and search for 'Disk Defragmenter'.
2. Click 'Analyze disk' to check your C: Drive's fragmented rate.
3. If the fragmented rate is above 10%, Windows recommends to defragment your C: Drive. As you can see below, mine had a whopping 48% fragmentation rate. 😨 My poor computer had never been defragged in 2.5 years of use.

46% Less Fragmented Disk Space After Two Defrags

Running the defragmenter once reduced my drive's fragmentation from 48% to 32%. Re-running the defragmenter dropped my C: drive to a 2% fragmented rate. That's more like it. 🤓

Fragmentation makes your hard disk do extra work that can slow down your computer. Removable storage devices such as USB flash drives can also become fragmented. Disk Defragmenter in Windows rearranges fragmented data so your disks and drives can work more efficiently.

Source: Ways to Improve Your Computer's Performance

My Windows 7 System Improvement Results

• Raised Windows Experience Index base sub-score from 4.9 to 5.0/7.9.
• Added 40 GB of hard drive space thanks to Disk Cleanup.
• Patched operating system security vulnerabilities and all software is up to date.
• Fixed any misbehaving Windows products.
• Decreased fragmented drive space from 48% to 2%. Windows recommends keeping it under 10%.

On paper, that looks great. Hopefully it means less spinning lag wheels and programs not responding when you really shoulda saved that document...  We'll see.