Lo-Fi Python

Jun 11, 2022

A Quick Windows Command Line Tab Key Shortcut

You might know you can use the tab key to view the files in a folder one at a time on the command line. However, I recently stumbled into another way to help navigate folders with a lot of files in the command line. Let's say you're looking for a file named "stuff.txt".

  1. Type "s".
  2. Hit the tab key.

You just jumped right to the files that start with "s". If there are not many files, you might go straight to "stuff.txt". The point is that you can jump straight to the letter of your file rather than scrolling through a mountain of files by hitting the tab key 50 times. Once you get to the relative place in the alphabet of your target file, you can scroll forward in the file list with the tab key or move backward by pressing shift + tab.

Multi-Letter Example

Type "st" + tab to target more precise multi-letter sequences.

I felt a productive zen when I accidentally realized this little command line quirk can save a lot of time. Especially when you have a folder with 50 or hundreds of files. Of course, you can always type in the file name directly. Using the tab key is very handy to browse if you don't know the exact file name. This is a nice little trick to add to your bag of Windows command line skills!

command prompt typing example

Jan 10, 2022

Analyzing Messi vs. Ronaldo with the FIFA API + jq + curl

Who is the world's greatest footballer, Messi or Ronaldo? EA Sports surely has calculated the answer to this question in their player ratings. They rate peak Crisitiano Ronaldo, Lionel Messi and Luka Modrić at 99 overall, with Neymar and Lewandowski at 98. Anecdotally, Messi has won 7 Ballon d'Or, the highest individual football honor one can achieve each year. Ronaldo has won 5 B'allon d'Or. Modrić has won 1 Ballon d'Or. Lewandowski was runner up this year, but has never won the honor. Neymar has never won a Ballon d'Or.

In FIFA, a player's video game representation is modeled intricately in a series of traits and specialties characterizing each player. The "Ultimate Team" EA Sports API is viewable as a plain json page or more cheekily with one line of curl and jq, a "command line json processor":

curl 'https://www.easports.com/fifa/ultimate-team/api/fut/item' | jq '.'

Enter this in a shell or command line. The result is beautiful, readable, pretty printed json!

Messi (Top) Vs. Ronaldo (Bottom) FIFA Player Ratings

These ratings represent the players at their peak of their careers. Messi is a better dribbler, while Ronaldo has more power and strength. Messi has the edge in free kicks, curve in his shot and "longshots" 99 to 98 over Cristiano. They are tied at "finishing", each with 99. Ronaldo has the "Power Free-Kick" trait, whereas Messi has "Chip Shot", "Finesse Shot" and "Playmaker" traits giving him an edge.

EA's ratings suggest that both are prominent goal scorers, with a slight edge to Messi in finesse and shooting from distance. However, there's something to be said for kicking the ball really damn hard. Ronaldo has superior raw shot power and a lethal combo of more powerful jump and stronger headers. All this combined with an "Aerial Threat" specialty enables Ronaldo to vault above and around defenders to smash in golazos off the volley. Ronaldo sizes up to 6' 2" (187 cm) vs. Messi's 5' 7" (170 cm) frame. This Portugese man definitely has an advantage in getting higher in the air. But the Argentinian is quite darty.

Messi has incredible accuracy from distance. He's also a better passer all around and has perfect "vision", great qualities for winning football games. Only in crossing does he have a lower passing rating. Ronaldo is also 10 points better at "penalties" or penalty kicks. The closer he gets to the goal, the more dangerous he is. Messi is more dangerous with the ball while dribbling, passing or shooting except when taking a PK.

Advantages can be gained in many different aspects of soccer. EA has developed a fun dataset to model these all time greats across several football skill dimensions. In 2022's version of the game, Messi is rated a 93, with Cristiano 91. Clearly these two are worthy of top honors. Don't forget Robert Lewandowski, with a 92 rating, who consistently lights up the Champions League and Bundesliga.

jq ftw

I had never used jq before this. Really enjoyed the quick, stylish and practical view of some json. This cool terminal display and syntax highlighting was on my Chromebook shell. It's neat how easily you can pretty print json with jq. I rate it a 99 for json pretty processing and pretty printing on the FIFA scale. Read more in the jq documentation!

May 06, 2020

Script Windows Like A Pro: Command Line, Batch Files, Remote Desktop Connection and pywin32

Here are a few useful corners of the vast array of Windows scripting tools.

A Few General Windows Commands

Use find to look in a text file to count the lines matching a string:

find /C "FAIL" < "Test_Results.txt"
# returns: 0 if no match or # of lines found, e.g. 2,50,100

I wrote a post on findstr, which offers similar functionality.

  • clip: pipe commands into the clipboard.
  • If: If Statements based on if files exist.
  • List ip address-related info:
ipconfig

Check system bit (usually 64-bit or 32-bit):

wmic os getosarchitecture

Automate Windows Scripts with Batch Files

Batch files can be run from command prompt or by double-clicking them. Here's an example of text in a batch file that activates a python virtual environment. Swap in your username and environment if you've created it.

cmd /k "cd C:\Users\your_username\PythonEnv\Scripts & activate & cd .. & dir"
  1. Save above as a .bat file.
  2. This uses cmd to open a new command prompt in a Windows batch file.
  3. cdinto my python virtual env then activate it by running a batch file.
  4. Then call dir to print directory contents.

Set a custom system 'last_name' variable to be recalled later.

set /p last_name=Enter a last name:
echo %last_name%
pause

Here we print it out with echo. Then pause.

Line continuation in batch files: Use ^ to continue your batch file scripts on a new line.

System Assessment Tools: powercfg and sfc

Display system stats:

systeminfo

Use powercfg to assess power, sleep and system states

powercfg /SLEEPSTUDY

Use sfc to perform a system file check:

# scan and repair
sfc /SCANNOW
# scan, but do not repair:
sfc /VERIFYONLY

Accessing a Remote Computer From the Command Line

You may want to ping a remote computer to see if it's running. Add your ip address instead of the below 1s and 0s:

ping 01.10.10.01

Log into your Remote Desktop with mstsc:

  1. Run Remote Desktop Connection, save an RDP file from Windows Desktop Client.
  2. You may need to adjust your credentials on your local machine.
  3. Finally, trigger login to an active window from command prompt:
mstsc RDP_File_Name.rdp

WinRM and WinRS can allow terminal access to your Remote Desktop. You may need to set your wifi network to private. To configure winrm:

winrm quickconfig

Log into a remote computer with winrs and run ipconfig:

winrs -r:https://myserver.com -t:600 -u:administrator -p:$%fgh7 ipconfig

Check Out Python's pywin32 Module

This module is extremely useful for scripting out Windows applications. For example, I've made good use of its interfaces to Outlook and Task Scheduler. Install with pip:

python -m pip install pywin32

Here's an example to send an Outlook email:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
import win32com.client

outlook = win32com.client.Dispatch('outlook.application')
mail = outlook.CreateItem(0)
mail.To = '[email protected]'
mail.CC = '[email protected]'
mail.Subject = 'Moneyball Review'
mail.Body = """Moneyball is an inspiring movie, based on real events.
            Brad Pitt, Jonah Hill and Philip Seymour Hoffmann gave great performances.
            The trade deadline scene is delightful. Wow.
            Chris Pratt as Hatteberg too. What a solid film.
            Money isn't everything. Playing ball is.
            """
mail.Attachments.Add('Baseball_Analysis.csv')
mail.Send()

Jan 20, 2018

How to Install Libraries and Enable the pip Installer in Python

Python comes with a bunch of standard modules. My favorites are shutil, glob, datetime, time, os (operating system), re (regular expressions) and webbrowser. The standard library is loaded.

Inevitably, you'll want to install new libraries from Python's rich ecosystem of external modules. Enter pip, Python's handy package manager and people's champion.

This post will teach you some Python history, show how to install pandas, and help you troubleshoot problems if it's not working. You'll find Windows and Linux commands for venv setup (recommended). With pip, you'll feel like Neo when installing new modules. Any skill is at your fingertips. It's like learning kung fu. There's probably a library for that!

I know kung fu

First, Some Python Version Caveats + History

Python 2 reached end of life on January 1st, 2020. Python 2 has officially been sunset.

Python comes with pip now, no setup is required. But certain versions such as Python 3.2 or the Python 2.7 that came stock on my improbably still functioning 2008 black Macbook, for example, may not have it installed.

In December 2021, Python 3.6 reached "end of life phase". Python 3.6 is "now effectively frozen". Read more in PEP 494. (Released Oct. 2022)

TLDR: use Python 3.7 to 3.11. This blog endorses using the lightning fast Python version 3.11.

Enter This in Your Terminal

python -m pip install pandas

Pandas is a super useful library for wrangling spreadsheet data, AKA "tabular" data. If successful, you should see activity that looks similar to the below screenshot, where I am installing openpyxl, an additional Python Excel library you'll likely want. You are good to go! This is the part where you get to feel like Neo! See Installing Python Modules in the Python Documentation for more detailed instructions.

neo_pip

To view all your installed libraries, enter:

pip list

Write a "requirements.txt" of installed libraries:

pip freeze > requirements.txt

You can list your outdated packages with the --outdated argument:

pip list --outdated

Use pip's -h help command line argument:

pip -h

View your system and user pip config settings:

pip config debug

Supplementary Resources

Congrats on figuring out how to install packages with pip, have fun!

Having issues? Try upgrading your pip version.

python -m pip install --upgrade pip

Try the ensurepip command.

This command will install and upgrade pip to the newest version. New in Python 3.4:

python -m ensurepip --upgrade

"The ensurepip package provides support for bootstrapping the pip installer into an existing Python installation or virtual environment. This bootstrapping approach reflects the fact that pip is an independent project with its own release cycle, and the latest available stable version is bundled with maintenance and feature releases of the CPython reference interpreter."

- ensurepip Python Documentation

You should follow best practice and create a virtual environment before installing libraries. venv or virtualenv. To create with venv:

python3 -m venv add_env_name_here

After your environment is created, activate it with the first command below, then install a library on Ubuntu Linux:

source add_env_path_here/bin activate
python -m pip install pandas

Alternatively, on Windows computers:

cd add_env_path_here\scripts & activate
python -m pip install pandas

Getting the prefix right can be tricky.

In the install command, the prefix is a reference to your Python executable. You may just need to alter your prefix to call it correctly. Here are some to try in place of "python". Observe what happens when you run these command variations. Good luck!

python3 -m pip install pandas
python3.11 -m pip install pandas
py -m pip install pandas
pip3 install pandas

How to Manually Enable the pip Installer

The rest of this post may be useful to you if you are:

  1. Working on legacy Python 2 or < 3.3 for which pip is not installed.
  2. Seeking to fix a faulty pip install that is not working properly.
  3. Curious to know how to manually set up pip.

Assumes Python is already installed. If you're running Windows 10, I found it easy to install Python from the Windows store. Download the get-pip.py file. Go to the link, right click the page and "Save As" a .py file to download. Then place the file where you want to access it. I placed mine in C:Python27Libsite-packages

You could also download the file with curl:

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.pyt-pip.py

If you are not sure where your site-packages folder is, type python -m site into command prompt for file path ideas.

Run the get-pip.py file.

Using command prompt's cd command with a Windows "&" operator to run the Python file in a Windows command prompt:

cd c:\Python27\Lib\site-packages & python get-pip.py

Or Linux terminal:

cd /Python27/Lib/site-packages && python get-pip.py

You should see some activity in command prompt that shows installation/updating of "setup" and "wheel". When it finishes, you have installed pip.

Type into command prompt at the same location:

python -m pip install requests

This installs the Requests module into your Python libraries. Requests is an http module which is highly regarded almost universally by the Python community.


Thanks for reading!
Check out these other posts with pip installed library examples:

fix Grammar and Spelling with language_tool_python and textblob

static site generation with pelican

text mojibake mash fixing with ftfy

a guide to making HTTP requests

simple GUI for scripts with gooey

Aug 09, 2017

Creating A Simple Website and Server Environment with Node.js and Express.js

Here is what I have deduced is the fastest way to get an app up and running with Node.js. This requires some familiarity with using the command line. I completed the Codeacademy course "Learn The Command Line" before beginning with Node.js. I think it helped me better understand what the commands are and what they do.

Download and install Node.JS Open the node command prompt. This was done on a windows machine.

First, create a folder for your app(s):

mkdir node_apps

Change the command prompt directory to your app's folder:
cd \app_name

Creates json file for your app. Fill out applicable info or just hit enter until the file is created.
npm init

Install express.js module in node.js:

npm install express

Install express-generator module in node.js:
npm install express-generator -g

Create the structure for your app and all necessary folders. (views, css, Javascript, routing, etc.)

express app_name

Ensure all app module dependencies are installed:

npm install

Start your server and web app:

npm start

Go to http://localhost:3000 in a browser. Port 3000 is the default port of Express. Your app is live in dev environment.

Notes

  • I learned most of this from this great blog post.
  • The above does not include a database integration. I integrated with a MongoDB database by following a blog post that has since been removed from the internet.
  • This YouTube video was also very helpful to me for figuring out MongoDB and Node.js integration.
  • An HTML shorthand language called jade (aka pug) comes stock within Express.js. Here's further reading on the pros and cons.
  • All of the above has been from my own studies. I do not claim anything listed as the most efficient or best way to use Node.js. This is what has worked for me over the past two days.
  • It feels good to whip up a nimble app environment that is capable of producing and supporting world changing software; Node.js is used by Netflix, PayPal, Microsoft and Uber.