Lo-Fi Python

Using The Debian Installer-Loader

1. First, back up your Windows computer files. Then download the Debian-Installer Loader Windows executable from the Debian wiki.
2. Click the downloaded executable and follow instructions. I followed the default settings all the way through.
3. You may need to choose your own partitioning settings to ensure Windows is preserved if desired.
4. During installation, choose your Linux collection. I chose Xfce because it seems to be highest ranked among Linux users and "not just helpful for older computers where few system resources are available, but also simply for those who want to get the most out of their systems." Gnome and KDE are other popular options.

After completing installation, restart your computer and select your new OS on boot-up. The following error codes displayed for me while starting up, signaling missing wifi firmware.

ERROR Failed to load firmware!
b43ssb0:0: firmware: failed to load b43/ucode5.fw (-2)
b43ssb0:0: firmware: failed to load b43-open/ucode5.fw (-2)
b43-phy0 ERROR: You must go to https://wireless.wiki.kernel.org/en/users/drivers/b43#devicefirmware and download the correct firmware for this driver version.

It took me a few boot-ups before I realized what this error message meant. In the rest of this post I am trying to figure out and fix what is wrong before I saw the error message. I enjoyed learning how to introspect Linux networks, but if I were trying to fix this problem again, I'd go to directly to *`this page <http://linuxwireless.sipsolutions.net/en/users/Drivers/b43/>`__*, which is linked to from the link in the error message, and try the solution there first.

Post Installation Setup

Open up the terminal once you're into your new Desktop OS and enter the below commands.

su -
apt-get install sudo -y usermod -aG sudo yourusername

1) Enable yourself as root user.

2) Install sudo.

3) Give yourself sudo user permission.

Optional: Replacing Network-Manager With Wicd

This Debian 9 package ships with Network-Manager. After logging in, I wasn't sure why wifi was not working, so I decided to remove Network-Manager and install Wicd. (This was before I realized what the error code displayed on boot-up meant.) Wicd is a Linux network managing alternative and it's built with Python, by the way. I followed these instructions to execute the below commands.

uninstall nm
sudo apt-get install -d --reinstall network-manager network-manager-gnome
install wicd
sudo apt-get install wicd-gtk

After installing Wicd, my Ethernet connection was not working. This fixed it for me:

sudo ifconfig eth0 up

Troubleshooting Linux Wifi & Inspecting Your System

Now, let's check for enabled network interfaces. "wlan0" is usually the name of the wireless interface. Does wlan0 show when you enter this command? If not, then you may need to update your wifi firmware. This was the case for me. Below is an output where wlan0 is correctly configured.

sudo ifconfig

eth0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
ether 00:25:a5:cf:38:7d txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 17
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10
loop txqueuelen 1 (Local Loopback)
RX packets 4 bytes 240 (240.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 4 bytes 240 (240.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
wlan0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 191.142.1.297 netmask 255.255.255.0 broadcast 182.138.5.255
inet6 2601:241:8c00:50ea:21a:92ff:fe0d:7531 prefixlen 64 scopeid 0x0
inet6 fe80::22a:42tf:fe0d:7531 prefixlen 64 scopeid 0x20 ether 00:2a:92:2d:45:51 txqueuelen 1000 (Ethernet)
RX packets 8509 bytes 4639778 (4.4 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 6206 bytes 923792 (902.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

Check wlan0 is not blocked on kill list.

sudo rfkill list all

Some computers have a "kill switch" for wifi. This command lists any blocked interfaces. If it is blocked, this thread might be useful. If nothing shows when you run this command, or if you see wlan0 is not blocked, carry on.

Check which wifi controller you have. This thread provides more info on what this means.

lspci -nn | grep -e 0200 -e 0280

03:00.0 Ethernet controller [0200]: Broadcom Limited BCM4401-B0 100Base-TX [14e4:170c] (rev 02)
0b:00.0 Network controller [0280]: Broadcom Limited BCM4311 802.11b/g WLAN [14e4:4311] (rev 01)

Find your system architecture. This determines which firmware you should download in the next step.

sudo dpkg --print-architecture

First, read here to determine the right packages for your Linux system. Then download the appropriate missing wifi firmware. For Debian, I downloaded the two packages below.

1. b43-fwcutter 2) b43-installer

"cd" into the directory with .deb files. Run the below commands to install the new firmware, then reboot your computer. The last two commands are adapted from this thread.

sudo dpkg -i firmware-b43-installer_019-3_all.deb
sudo dpkg -i firmware-b43-fwcutter_019-3_i386.deb
sudo modprobe -r b43
sudo modprobe b43

Edit Wicd preferences to set wlan0 as the wireless interface if needed.

Success! Wireless networks are now showing.

Wrapping Up

I'd like to thank the awesome people who contributed to the Debian Installer-loader and all the help in Linux forums that enabled me figure this out. I'm new to the world of Linux but already enjoying diving into this operating system. Its ability to do just about anything from the command line are a lot of fun. I am now running two Linux systems, one on my Dell and another running Ubuntu that I installed on a Chromebook with Crouton. Both have been relatively painless to set up. It this case, it turned a sluggish laptop into a very capable machine. They should call it Lit-nux :)

Full Disclosure

This worked on my computer for a few days before the keyboard stopped working correctly on my computer. Typing became impossible because the keys didn't work or entered the wrong letters when pressed. I'm not sure what the cause of it was, but consider that before attempting this on a machine. Be prepared to lose it. If you really need the machine to be functional, it may not be a great idea to try this. This was attempted on an old beat up computer. I would try this method of porting a Windows machine to Linux again as a salvage project or on a low-risk Windows machine if I had one lying around.

The first 5 scripts use modules that ship with Python.

(1) Uppercase all of the file names in the current working directory.

1
2
3
4

import os
filenames = os.listdir(os.getcwd())
for filename in filenames:
    os.rename(filename, filename.upper())

os module documentation

(2) Get all filenames in the current directory and write to a text file.

1
2
3
4
5

import os
folder_contents = os.listdir(os.getcwd())
with open("list_of_dir_files.txt","w") as fhand:
    for item in folder_contents:
        fhand.write('f{item}\n')

(3) Check what day of the week is today.

1
2
3
4
5
6
7

import calendar
from datetime import date
today = date.today()
if calendar.day_name[today.weekday()] == 'Friday':
    print("Today is Friday.")
else:
    print("Today is not Friday.")

calendar documentation

(4) Get the two most recent file names from a directory. This is a Windows file path example. I have escaped the backslashes below to make it work in my Atom text editor.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

import glob
import getpass
import os
# getctime == by created date, getmtime == by date modified
# get all files, regardless of extension
folder = f'C:/Users/{getpass.getuser()}/Desktop/'
list_of_files = glob.glob(folder)
newest_file = sorted(glob.iglob('C:/Users/your_username/Desktop/*.csv'), key=os.path.getmtime)[-1]
second_newest_file = sorted(glob.iglob('C:/Users/your_username/Desktop/*.csv'), key=os.path.getmtime)[-2]
print(newest_file, second_newest_file)

glob module documentation: https://docs.python.org/3/library/glob.html#glob.iglob

(5) Auto-Hamilton Ticket Lottery

1
2

import webbrowser
webbrowser.open('https://www.luckyseat.com/hamilton-chi/', new=0)

webbrowser PMOTW: https://pymotw.com/3/webbrowser

To Run The Next 3 Scripts, install pandas:

python -m pip install pandas

(6) Split a csv file into smaller excel files.

1
2
3
4
5
6
7

import pandas as pd
file_name = 'big_file.csv'
# slice off extension
src_file_minus_ext = file_name.split('.')[:-1]
for i, df in enumerate(pd.read_csv(file_name, chunksize=250000)):
    csv = f'{src_file_minus_ext}{str(i)}.csv'
    df.to_csv(csv, index=False, header=df.columns)

pandas beginner tutorial: https://www.youtube.com/watch?v=5JnMutdy6Fw

(7) Convert a .xlsx to .csv

1
2
3

import pandas as pd
df = pd.read_excel("input.xlsx")
df.to_csv("output.csv", index=False)

(8) Convert a .xlsx to .html

1
2
3
4

import pandas as pd
file_name = "data.xlsx"
df = pd.read_excel(file_name)
df.to_html("data.html")

To Run The Last Two Scripts, install pdfkit:

pdfkit relies on another library, wkhtmltopdf, which adds an extra wrinkle to setting it up for Windows users. I've laid out the differences between using pdfkit for Windows vs. Ubuntu below.

python -m pip install pdfkit

installing wkhtmltopdf on Windows

To use pdfkit on Windows, go to the link, choose your version to download. This installs a Windows executable that pdfkit needs to find to work.

https://wkhtmltopdf.org/downloads.html

installing wkhtmltopdf on Ubuntu

sudo apt-get install wkhtmltopdf


**For Windows, **\ `download wkhtmltopdf <https://wkhtmltopdf.org/downloads.html>`__\ ** and add this config patch for pdfkit:**

1
2
3
4

import pdfkit
url = 'https://www.hollywoodreporter.com/news/grammys-alicia-keys-best-moments-as-host-1185013'
config = pdfkit.configuration(wkhtmltopdf='C:\\Program Files\\wkhtmltopdf\\bin\\wkhtmltopdf.exe')
pdfkit.from_url(url, 'webpage.pdf', configuration=config)

4 Convert a .html file to .pdf. (Add config patch above for Windows.)

1
2

import pdfkit
pdfkit.from_file('data.html', 'report.pdf')

5 Create a pdf snapshot of a webpage on Ubuntu below. (Add config patch above for Windows.)

1
2
3

import pdfkit
url = 'https://www.hollywoodreporter.com/news/grammys-alicia-keys-best-moments-as-host-1185013'
pdfkit.from_url(url, 'alicia_keys.pdf')

pdfkit documentation: https://pypi.org/project/pdfkit/

Final Thoughts

I am very pleased with pdfkit. I am also toying around with Python's PyFPDF: https://pyfpdf.readthedocs.io/en/latest/index.html and PyPDF2: https://pythonhosted.org/PyPDF2 libraries. Together, these three offer some unique abilities for creating and manipulating pdf files. And the pandas library is amazing for reading and writing tabular data sheets, can't say enough great things about pandas. Worth the time. Hope you enjoyed these Lightning Scripts :)

Enter This in Your Terminal

python -m pip install pandas

Pandas is a super useful library for wrangling spreadsheet data, AKA "tabular" data. If successful, you should see activity that looks similar to the below screenshot, where I am installing openpyxl, an additional Python Excel library you'll likely want. You are good to go! This is the part where you get to feel like Neo! See Installing Python Modules in the Python Documentation for more detailed instructions.

To view all your installed libraries, enter:

pip list

Write a "requirements.txt" of installed libraries:

pip freeze > requirements.txt

You can list your outdated packages with the --outdated argument:

pip list --outdated

Use pip's -h help command line argument:

pip -h

View your system and user pip config settings:

pip config debug

Supplementary Resources

• Take a look at this list of 20 modules to get started. Here is another more comprehensive list of libraries you can now install.
• Swing by the pip documentation Quickstartand User Guide to learn some helpful commands.
• PyPI, the Python Package Index is the official Python package repository.
• Why you should use 'python -m pip'
• pip cheat sheet from opensource.com

Congrats on figuring out how to install packages with pip, have fun!

Having issues? Try upgrading your pip version.

python -m pip install --upgrade pip

Try the ensurepip command.

This command will install and upgrade pip to the newest version. New in Python 3.4:

python -m ensurepip --upgrade

"The ensurepip package provides support for bootstrapping the pip installer into an existing Python installation or virtual environment. This bootstrapping approach reflects the fact that pip is an independent project with its own release cycle, and the latest available stable version is bundled with maintenance and feature releases of the CPython reference interpreter."

- ensurepip Python Documentation

You should follow best practice and create a virtual environment before installing libraries. venv or virtualenv. To create with venv:

python3 -m venv add_env_name_here

After your environment is created, activate it with the first command below, then install a library on Ubuntu Linux:

source add_env_path_here/bin activate
python -m pip install pandas

Alternatively, on Windows computers:

cd add_env_path_here\scripts & activate
python -m pip install pandas

Getting the prefix right can be tricky.

In the install command, the prefix is a reference to your Python executable. You may just need to alter your prefix to call it correctly. Here are some to try in place of "python". Observe what happens when you run these command variations. Good luck!

python3 -m pip install pandas
python3.11 -m pip install pandas
py -m pip install pandas
pip3 install pandas

How to Manually Enable the pip Installer

The rest of this post may be useful to you if you are:

1. Working on legacy Python 2 or < 3.3 for which pip is not installed.
2. Seeking to fix a faulty pip install that is not working properly.
3. Curious to know how to manually set up pip.

Assumes Python is already installed. If you're running Windows 10, I found it easy to install Python from the Windows store. Download the get-pip.py file. Go to the link, right click the page and "Save As" a .py file to download. Then place the file where you want to access it. I placed mine in C:Python27Libsite-packages

You could also download the file with curl:

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.pyt-pip.py

If you are not sure where your site-packages folder is, type python -m site into command prompt for file path ideas.

Run the get-pip.py file.

Using command prompt's cd command with a Windows "&" operator to run the Python file in a Windows command prompt:

cd c:\Python27\Lib\site-packages & python get-pip.py

Or Linux terminal:

cd /Python27/Lib/site-packages && python get-pip.py

You should see some activity in command prompt that shows installation/updating of "setup" and "wheel". When it finishes, you have installed pip.

Type into command prompt at the same location:

python -m pip install requests

This installs the Requests module into your Python libraries. Requests is an http module which is highly regarded almost universally by the Python community.

fix Grammar and Spelling with language_tool_python and textblob

static site generation with pelican

text mojibake mash fixing with ftfy

a guide to making HTTP requests

simple GUI for scripts with gooey