Lo-Fi Python

40 Open Source Libraries and Tools for the Modern Developer

I usually find a lot of these tools from viewing GitHub's trending repositories. When I find a novel repo, I'll star it to remember for later. This is a list of free, open source software spanning security, AI, cloud, HTTP, JSON, monitoring, dev ops and more.

I enjoy finding and cataloging new packages as much as using them sometimes. Some of these I'll never use. By filing them away, I can recall in it the future if I am faced with a problem that they can solve. Enjoy!

1. NeMo: a LLM (large language model) toolkit for conversational AI from Nvidia
2. LMOps: general technology for enabling LLMs from Microsoft
3. Prometheus: open source monitoring tool, built on top of Go
4. aws-cli: universal command line AWS interface
5. httptoolkit: "HTTP Toolkit is a beautiful & open-source tool for debugging, testing and building with HTTP(S) on Windows, Linux & Mac"
6. curl: ubiquitous command line tool for transferring data with URL syntax
7. curl-impersonate: a special build of curl that can impersonate Chrome and Firefox browsers
8. jq: touted JSON processor command line tool
9. jc: CLI tool and Python library that converts out of popular command line tools to other formats
10. JSONView: a Chrome web extension that applies syntax highlighting and folding to JSON in your browser, written in Typescript
11. thug: a Python based "honey client", used to mimic user behavior in a web browser in order to detect and emulate malicious content
12. OpenTelemetry: open source tracing and monitoring app
13. nocodb: an open source airtable alternative
14. dnstake: a fast tool to check for DNS vulnerabilities
15. croc: easily and securely send things from one computer to another
16. buku: inter-browser bookmark transfer tool
17. backstage: an open platform for building developer portals, written in Typescript
18. sentry: a "developer first" error tracking and performance monitoring platform
19. sidekick: free and open source live action debugging platform, like chrome dev tools for your backend
20. Prowler: an open source security tool to perform cloud best practices
21. Earthly: create a CI/CD continuous developer system
22. metabase: the easy, open source way for everyone in your company to ask questions and learn from data
23. Zulip: open source team chat
24. Joplin: an open source note taking and to do list application
25. fastup: a tool for gaining insights from large image collections and detecting anomalies
26. Upptime: an open source uptime monitor and status page connected to Github
27. Rembg: tool to remove the background from images
28. dyna-cli: convert Python functions to Linux shell commands
29. Penpot: the open source design & prototyping platform
30. pgcat: postgresQL pooler with sharding, load balancing and fail over support
31. pedalboard: a Python library for working with audio from Spotify
32. OpenRPA: free open source enterprise grade robot process automation software. See also: more free RPA tools
33. hook-slinger: manage webhooks with Python
34. vidcutter: a modern video editing tool
35. XSSstrike: most advanced Cross Site Scripting detector
36. wifiPhisher: wifi security testing tool with Python extensions
37. Salt: an "any infrastructure" automation tool built on Python
38. Locust: a scalable load testing Python library
39. Fabric: simple, Pythonic remote execution and deployment via ssh
40. ShareX: free and open source program that lets you capture and record your screen, written in C#

"Shutdown" a Windows Computer by Double-clicking a Batch File

Here is a quick and easy way to automate turning off your computer. This saves me about 15 seconds to manually click the start menu and restart buttons. It worked for me on an old, laggy HP computer running the Windows 10 operating system. Now, I can double-click a batch file on my Desktop and walk away while it struggles.

Batch files are executable via:

• double-clicking them
• right-clicking and selecting "Run"
• entering the batch file name in command prompt, ex: "shut down CPU.bat" if the current working directory is in the same folder as the batch file

Open a blank Notepad document and save as shut down CPU.bat with this text:

cmd /c shutdown -s

Source: Microsoft shutdown documentation

When this batch file runs, it will trigger a pop-up window warning that your computer is about to shut down. For my slow, slogging computer it shut off about 20 seconds later. This may also trigger queued automatic updates to install, which happened when I used the above command.

Source: Stack Overflow

App Review: Why You Should Use Pocket to Save Your Reads

Update: Mozilla has chosen to sunset its Pocket app. As of July 8th 2025, Pocket will be shut down.

I really enjoyed using Pocket as my primary reading app on my cell phone. My top need is to download articles I find on the internet for offline viewing. This app does that and more in a smooth interface. This review reflects the perspective of using the Pocket Android version 8.1.1.0, pictured here:

dark theme with saved articles

Key Features and Benefits

• download articles & blogs to read offline
• quickly share a webpage to the Pocket app in 2 taps from your browser
• dark or light themes available
• free app, downloadable on your Android or iOS device
• converts and stores articles as podcast style listenable audio
• ability to archive or delete old articles
• includes tags to categorize your reads
• filter by length of the read, "short reads" or "long reads" and read or unread
• the length of the article is displayed in minutes it takes to read it
• chooses the best format to present the website, as "article view" or "web view"
• auto-bookmarking, aka you can close out of an article and it will remember where you were the next time you open it
• discover new stuff to read within the Pocket app
• highlight key points in a text to remember and save for later
• For developers, there is a Pocket API. Apps and integrations are available for other products like Zapier, Evernote, Amazon Alexa and Slack.
• created by Mozilla, a privacy friendly company that also created the Firefox web browser

saving is quick and easy

filter saved articles by length

The Pocket app is an overall great product and I felt compelled to endorse it here for the satisfaction it brought me. It has increased my reading throughput. It was perfect for when you found yourself without an internet connection, in sparsely populated areas, on an airplane or while traveling in foreign countries. Goodbye, pocket. You will be missed.

Getting People to Use Software is Hard

You're sure you've created a stellar solution to a problem at hand. You took the time to think an idea out and execute. You iterated tirelessly, making tweaks and creating a tool that will sell itself to potential human users. Everyone loves their own creation. Others will realize the tool's value too!

There's only one problem. People's default setting is to not want to use your software. Whether it's low code, no code, Excel, PHP or Python driven, people are resistant to software. They don't want to use it unless it's so obvious and easy that the value is immediately visible.

Image Source

Getting humans to put in the time and effort to learn how to use your tools is really tough. It has to be excellent. The bar is high. You have to understand your users' needs and perceptions intimately. And even then, you'll probably still fail. Regardless, we keep trying because we are obsessed with solving a problem or simply paid to keep trying to solve the problem.

Building a solution is easy. Automating a task to maintain application state is also sometimes. In my experience, making something that other humans will actually trust and use is not. Keep trying. /rant

6 of the Hottest Python Libraries in 2023

Here's a short list of Python packages making a splash this year:

• pyscript: execute Python code in your web browser
• ruff: code linting package built with Rust and Python
• FastAPI: a rapidly adopted web framework for building APIs
• polars: high performance pandas drop in replacement, also built with Rust and Python
• buzz: Python package that accesses the Whisper API's text transcription of audio
• tomllib: new in the Python 3.11 standard library. It's quietly picking up steam amongst Python developers. Tom's Obvious Minimal Language, TOML, "a config file format for humans"

Python + Rust = High Performance

A common theme I've noticed is the emergence of Rust as a performance complement to Python code. It seems we can expect modularity between the two languages to strengthen their collective abilities.

example polars dataframe code

Python is still growing and evolving.

It's great to see. Including the ability to parse TOML config files in the standard library is a vote of confidence by the Python community. I will now seek to use TOML in my own projects.

Web frameworks and web browsers are king.

FastAPI is the new option in a space dominated by Flask and Django. Web frameworks allow developers to quickly create websites.

Running Python in a browser has been a movement in recent years. First Pyodide, WASM euphoria, and now a Python library that takes Python in a browser to new levels. Excited to see where these projects go and what new buzzy libraries will emerge this year!

pymarketer: an HTTP + Spreadsheet Wrangling Python package

Typically, this blog reviews the other Python libraries in its vast ecosystem. This time, it's my own package I made for fun, pymarketer. This was created in a single day and can be installed from the Github repo. Have a go at my most read post if you need help with pip.

Install with pip from the source Github repo:

python -m pip install git+https://github.com/erickbytes/pymarketer.git

The pymarketer package helps you do things like:

• merging all the tabs of an Excel file into one CSV
• generate HTTP code
• make a word cloud image
• splitting a CSV
• merging CSVs

Generating a Word Cloud with the pymarketer Package** via wordcloud

1
2
3
4
5
6
7
8

import pandas as pd
import pymarketer as pm

xl = "Chicago Breweries.xlsx"
df = pd.read_excel(xl)
# Make a wordcloud from a pandas dataframe.
wordcloud = pm.word_cloud(df)
wordcloud.to_file("Text Word Cloud Visualization.jpg")

This package relied on several Python libraries to complete:

• chardet
• pandas
• numpy
• wordcloud
• ftfy

I'll likely expand on this in the future. Anyone who wrangles data might be able to apply this package to good profit. At minimum, you might find it interesting to take a look at the project's __init__.py to see how some of the functions are implemented.

Additional Resources

• How to Make an Internal Python Package
• pymarketer Github Examples

An Example Pelican + Git Blog Post Workflow

In my Ubuntu Linux environment, I'm now publishing new blog posts following this Pelican + Python + git workflow.

1. Activate the Python environment:

# Create with a virtual env with venv: python -m venv env_name
source env_name/bin/activate

2. Clone your repo and go to the project folder:

git clone https://github.com/erickbytes/lofipython.git && cd lofipython && ls

3. Run Python script to create new markdown or .rst file from a template:

python new_post.py

4. Compile the new post with the Pelican content command:

pelican content

5. Use the Pelican listen command to serve blog to the default port 8000:

pelican -l

6. Preview your new post at localhost:8000 in Firefox:

firefox -new-tab http://127.0.0.1:8000

7. Use git to stage, commit and push the files to a Github repo:

git add .
git commit -m "new post edits and fixes"
git push -u origin main

The new blog post is now live! This is my own workflow for my Pelican blog, this blog which is hosted for free with Cloudflare Pages. You can read more about connecting Pelican and Cloudflare in this past post I wrote.

Github SSH Required

You will need to create a SSH key and connect it to your Github account to get this working, as it's required by Github now. Make sure you write down your passphrase. I was able to create an ssh key with this command:

ssh-keygen -t ed25519 -C "yourname@example.com"

Scripting New Post Creation

Below is the short Python script I wrote for generating the markdown file for a new post, "new_post.py" in the above workflow.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

from datetime import date


def post_name():
    """Ask for the new post's name."""
    name = input("Enter the new post's title:\n")
    return name


def post_template(title):
    """Return str, post draft"""
    post = f"Title: {title} \nDate: {date.today()} 4:20 \nCategory: Essay"
    return post


def save_draft(name, post):
    """Save new post draft to content folder."""
    content = "~/projects/lofipython/content"
    name = name.replace(" ", "-")
    md = f"{content}/{name}.md"
    with open(md, "w") as fhand:
        fhand.write(post)
    return None


name = post_name()
post = post_template(name)
save_draft(name, post)

I've enjoyed working this out on my new blog. I can easily edit, improve and fire off blog posts rapidly with this command line workflow.

How to Find and Open a Public Wi-Fi Login Page

Sometimes, we find ourselves on a public Wi-Fi connection that is "connected", but unable to get to the login page. While the login page is triggered automatically on most mobile devices, computers may not connect so easily. As usual, I turned to Google to find a way to get to a public Wi-Fi login page.

This helpful Zapier blog post outlines a few different approaches you can take. My solution below is one of the IP addresses they suggested trying. Decide if you need to edit your computer's network settings. Alternatively, you may just need to find the public login page. I was able to load my public Wi-Fi login page with this IP address in my web browser:

192.168.1.11

This is a fun little trick to keep in mind if you're having difficulty connecting to public Wi-Fi. Problem solved!

How to Check Github Repo Star Counts With Python

Snooping through my package list, I noticed the PyGithub library was installed. Its repo boasts "Typed interactions with the GitHub API v3". I googled the package, wanting to check in on the repos I profiled in an earlier post about static site generators.

I drafted the code below after noticing the repo.stargazer_count function in its documentation. This is neat to have if you want to keep tabs on a batch of repos, instead of tediously checking the Github web interface! If you're new to Github, the trending page is an easy way to find new, interesting repos.

Getting Started

1. You'll need to create a personal access token for your Github account. See the Github docs, "Creating a personal access token".
2. Install PyGithub and pandas:

pip install PyGithub
pip install pandas

3. Run the below code as a Python script.

python github_stars.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

import pandas as pd
from github import Github


def stars(repo, g):
    """Retrieve github repo star count.
    Accepts: str, repo "username/repo name",ex: "getpelican/pelican"
    Returns: int, github repo stargazers number"""
    repo = g.get_repo(repo)
    return repo.stargazers_count


# static site repos: http://lofipython.com/a-brief-summary-of-promising-python-static-site-generators/
urls = [
    "https://github.com/getpelican/pelican",
    "https://github.com/lektor/lektor",
    "https://github.com/eudicots/Cactus",
    "https://github.com/getnikola/nikola",
    "https://github.com/sunainapai/makesite",
    "https://github.com/hyde/hyde",
    "https://github.com/Anomareh/mynt",
    "https://github.com/staticjinja/staticjinja",
]
repos = [url.replace("https://github.com/", "") for url in urls]
g = Github("access_token")
counts = [(repo, stars(repo, g)) for repo in repos]
stars_df = pd.DataFrame(counts, columns=["repo","stars"])
stars_df.to_csv("Stars.csv", index=False)

On Linux, I was able to check the results of the CSV with the cat command:

I confirmed the API was accurate against the web interface in pelican's repo!

Github Repo Stargazer API Reference

Pelican Fix for "No valid files found in content"...

Pelican is a popular static site generator library in Python. I didn't know why my pelican blog was not working. I've used the "pelican content" command many times for my blog. This time, when attempting to write a new post on a new computer, I was getting this error where none of my posts were visible to Pelican:

1
2
3
4
5

WARNING  No valid files found in content for the active         log.py:91
         readers:
           | BaseReader (static)
           | HTMLReader (htm, html)
           | RstReader (rst)

Solution

Install the markdown library, which is stated in the pelican docs. This Github issue also provides some background on this warning.

pip install markdown

Now my "pelican content" command works!

pelican content