Lo-Fi Python

Reflections on 5 Years of Solving Problems with Python

Prior to learning Python, I had no programming experience. I worked in marketing for a book publisher and did not perform well at my job. It was not a good fit. They eventually fired me. As my previous job unraveled, I discovered Python and the Coursera course, Programming for Everybody (Getting Started with Python). Fortunately, that course jump-started me onto a path of learning and reading each day. My aim was to make my own website, a goal that I accomplished. I needed to know how the sausage was made.

Looking back from 2020, I can safely say Python changed my life. Because of it, I now have a fulfilling marketingdata-oriented career. I'm also grateful for the financial stability that came with it. I love to learn about the language and continue to improve my abilities to solve problems with new tools, not only Python.

Below are pieces of wisdom picked up from my experiences. They are the result of many hours of study, reading, mistakes, luck, toil and eventual glory.

These are thought-provoking adages and guidelines, not absolute truths in all cases.

1. Developing a habit of learning pays off over time, no matter what the subject is. It is an investment in yourself that compounds.
2. Follow your own curiosity. It's less important to compare what you know to others. Compare what you know today to what you knew yesterday. Don't worry about how long it takes to learn.
3. Watch educational or technical conference talks on sites like YouTube or InfoQ. Rich Hickey, Brandon Rhodes and David Beazley are some of my favorite speakers. Watch talks from all languages, not just Python. Often the concepts apply to any programming language.
4. Use an RSS reader. Anytime you find a good blog, subscribe by RSS or email to get new posts. I use the Feeder Chrome extension\Android app.
5. The Zen of Python contains a lot of wisdom. I like the concept of Explicit is better than implicit. This implies declaring your actions in written or oral fashion, providing additional context. Consider favoring easier to read solutions over clever one-liners. For example:
   • List comprehensions are useful and "pythonic", use them! But sometimes it's easier to use a for loop to hash out an idea. (Contrarily, avoiding the Initialize Then Modify pattern benefits those comfortable with comprehensions.)
   • Explicitly using keyword arguments versus positional arguments is another way to make your code easier to understand.
6. Can you explain the solution simply? If not, try to clarify your understanding or maybe there's a simpler way. In Python, there are often several ways to accomplish the same goal. But keep in mind the Zen of Python: There should be one-- and preferably only one --obvious way to do it. Although that way may not be obvious at first unless you're Dutch. Look for the obvious way. An example of this is string formatting. I've heard f-strings are the hot new way to do this now.
7. Don't be afraid to change course if things don't feel right. Ask yourself while coding, "Does this feel efficient?" Recently I was trying to format a json string so I approached it like I had in the past, by exporting the request from Postman and formatting the json string with python's format() built-in. But this time, the curly braces were confusing me, I was struggling and it wasn't working. I googled and around and saw python's json module and df.to_json() in pandas. They were a much easier and better-looking solution. But it still wasn't working. Finally, i used the Postman approach and f-strings to format a successful payload. The third try worked! F-strings are super nice and clean.
8. If you're stuck, there's probably a free online course or blog post that explains whatever is confusing you. Use the Googles. When in doubt, Google the error message.
9. Begin your project by writing a list of requirements. This often leads to good questions and cases that may need to be addressed. The book Code Complete 2 covers establishing project requirements in great detail, along with the other stages in the life-cycle of a software project. I'm really enjoying this book and highly recommend it.
10. Names are really important. Take time to think about a good name for your variables and functions. Also, name your scripts well. I name my scripts using action verbs. For example, my script that organizes accumulated files on my desktop into folders is named clean_desktop_files.py. When I see this script months later, its name reminds me the action the script is performing. I believe it's better to err on the side of longer, more descriptive names for variables and functions. It makes code easier to understand. But there is a trade-off with length to keep in check.
11. Moving a block of code into a function can abstract away repetitive code and increase its readability.
12. Each function should do one thing only. Follow the single-responsibility principle.
13. Train yourself to think in data structure conversions. The Python dictionary is very useful and can be converted to and from lists, tuples, sets, etc. I often find it more efficient to convert to a different structure to efficiently organize it. Usually I am googling things like "convert class object to python dictionary" because dictionaries are easy to work with or convert to other structures. The vars() built-in is great for converting objects to a dictionary. For example, once you have a dictionary, you might be able to solve your problem by converting it to a dataframe.
14. Use only the data you need. Reading in just the essential data helps avoid memory issues and hanging programs. In pandas, the usecols argument in pd.read_csv() is great for this. This creates a dataframe with 2 columns:

1

df = pd.read_csv('emails.csv', usecols=['name','email'])

15. Assume that if something is broken, it's because of something you've done. Start from the assumption that your code contains the bug and work outward by eliminating possibilities. Avoid jumping to quick conclusions. Instead, carefully consider possible reasons for why something is happening. Many times, I find my 2nd or 3rd hypothesis is actually true.
16. There will be times when you'll look at someone else's choices and wonder why they did things a certain way. Consider the possibility that they know more than you in this domain.
17. Beware of sequencing errors. Are your tasks, scripts or functions executing in an efficient order to reach your end goal? Look to unblock bottlenecks and correct chronological mistakes in your processes.
18. Before you send that email asking for help, go back and take another look. There's also no shame in asking for help. Be sure you proofread your email before sending.
19. Status code 200 does not guarantee your API request was successful. You may want to write a test to confirm success that doesn't rely on response status codes.
20. Unfortunately, testing gets shunned sometimes. Make it a priority. I enjoy writing pytest tests more than most other code. Why? Because tests confirm my scripts are working to some degree, detect bugs and provide a refactoring safety net.
21. Refactoring your code is a crucial step in making it better. Coming back to my code after a few weeks, months or years brings clarity, experience and a new perspective. It feels good to improve the quality of my old work.
22. Consolidate your tasks. Bundling things can save you a bundle of time! Identify redundant patterns and remove if possible. Observe yourself while working. Any repetitive manual process can probably be automated away. Recently, I figured out how to use a Windows batch file to instantly activate my Python virtual environment. It took me a few years of tediously pasting the cd and activate commands into command prompt every day to realize. Now it's a snap.
23. Stack Overflow is a useful resource. But the top answers may be outdated. Check the other less popular answers sometimes. Or...
24. Read the documentation! An updated or more elegant solution might be there. I recently found os.makedirs(path, exist_ok=True) in the os docs. I didn't know about the exist_ok argument. I was creating folders with a more complicated alternative from Stack Overflow for years. I use this way all the time now. In the same vein, if you need the local system username, the Python docs state getpass.getuser() is preferred over os.getlogin().
25. Write documentation explaining how to use your projects. Even if you can only muster a quick README text file, that's better than nothing. Within your code, docstrings are a nice addition. I have yet to use Sphinx, but it is a popular choice for generating documentation.
26. Teaching others feels good and solidifies your knowledge. Writing and pair programming are great ways to improve your understanding and pass your skills along to other people. While we're on the subject of writing...
27. Write everything down! Your head is not good at storing information in memory. Computers are. This frees your mind to come up with new ideas rather than expending energy to remember what you've already done. It also helps you plan. I use a Notepad text file to keep a running to-do list. You could also use services like Trello or Microsoft Planner. While writing code, use comments and docstrings conservatively for quick notes, clarifications or reminders. The important thing is to write it down somewhere.
28. When editing your writing, continually ask yourself, "Do I need this word or phrase?" for every word you write.

"Brevity is the soul of wit." - William Shakespeare (Hamlet)

29. Draw inspiration from culture, nature and professional disciplines outside of your own. Insights can be mined from anything. Don't dismiss a situation as mundane without first scanning for knowledge nuggets and gems.
30. Better solutions often come to me after gaining time and experience with a problem. Building software is an iterative cycle of adjustment towards consistently fulfilling the needs of those it serves in 100% of cases. In a perfect world, you'd never have bugs. But edge cases tend to pop up in ways you didn't think of when you first wrote a solution. There will also be projects where requirements or business rules change. Consider that possibility when you are designing your solution.
31. It's possible to find a job that you're excited about and genuinely enjoy the work.
32. Respect your craft, whether it's coding or another profession. A skilled carpenter needs precision, practice and focus to make something beautiful. Approach your craft with the same mindset and pride in making your best art.
33. We all have holes in our knowledge. Be receptive to other ways of thinking. The best way to learn is from other humans. Everyone has different backgrounds and experiences. I have never used object oriented programming, classes or certain command line tools like ssh. I have a loose understanding of these things but have not yet applied them to my projects. Working with paths (os and pathlib) still gives me fits sometimes. These are knowledge gaps that I want to fill in. Additionally, we don't know what we don't know. Try to illuminate the fog of your unknown.
34. Choosing to dedicate to learning Python is among the best decisions I've made.
35. Attitude is more important than intelligence. Anyone can learn to program, play guitar or fly an airplane. You can become an adept problem solver. Acquire an attitude to support your determination and persistence.

[caption id="attachment_2981" align="alignnone" width="959"] Brandon Rhodes: Stopping to Sharpen Your Tools - PyWaw Summit 2015[/caption]

I'll leave you with the 4 P's and 4 C's from my Programming for Everybody Coursera course graduation ceremony. Cultivating these principles will guide you to growing your education and finding a positive course in life:

4 P's: Passion, Purpose, Persistence, Playfulness

4 C's: Choice, Commitment, Connection, Completion

Thank you for reading and I hope this post helps you on your own educational journey.