Edge cases are a big problem in software engineering, one that I recently ran into while on a beta box was errors being thrown after a downgrade.

First a little background on the application. It is a django application running in a docker container via docker-swarm. Our application upgrade/downgrade process is to reduce the amount of containers running down to 0. Change the tag numbers in our .yml file and then scale them back up. This forces the containers to download the newly defined version. Note that our downgrade process is exactly the same we just update the a tag with an older version.

The process described above might sound correct to you, however, because we use migrations to structure our database the process is flawed. By replaceing the tag and just moving to older docker containers, the downgrade function inside our migrations never get ran. Instead the downgraded application has no idea about the migrations that existed in the newer version that does not in it's own. Thus resulting in an inconsistant database state with the version of application deployed.

My current solution to this issue is to run the migration downgrade for each app in our django project manually inside of the existing application container . To do this I start out with a diff of the code between the old version and the new version. I look for any added migrations and run the command below to have django south run through the downgrade methods until it reaches the latest migration in the old code. Then I perform the old migration method.

Here is a step by step list of the process.

1) Diff of code.

2) Write down oldest added migration number and app name

3) Exec into the application docker container.

4) Migrate backwards :

./manage.py migrate appname 0000

5) Exit out of container.

6) Scale the swarm down to 0 nodes.

7) Change tag number.

8) Scale the containers back up.

Do you have any suggestions or any improvements to this process? Comment below and link me to other articles you've read about this process. The more we know together the better our overall solutions can be.

1) Chunk Your App Into Pieces

• Turn everything into a MICROservice.

2) Modernize Access To Data

• Implement Data Access via API's or using a similar data store.

• Developers should not care what it looks like on the backend. They should be able to easly access and manipulate the data in a secure way using their favorite languages.

3) Replace Decaying Code

• Slowly replace chunks of your application one piece at a time.

Benifits:

• Consolidation of Virtual Machines.

• Easier to move services around.

• Scaling services is easy.

• Chatops

• Use Open Source Technology

• Changes Culture as getting stuff done is very rewarding.

This isn't nessesarly a good representation of the best python tools or packages, but the top I have used over the past year.

pyorc - An amazing package that wraps around Tesseract so that you can do optical character recognition on imag

tox - A wonderful package that you can use to automate and standardize python testing.

psutil - Cross platform library that exposes process and system information.

gmvault - Not a library, but still written in python. It gives me the ability to backup and restore my emails from gmail, locally.

elasticsearch-dsl-py - Favorite high level library for elasticsearch manipulation.

2to3 - Great tool for helping move from a python 2 codebase to a python 3 codebase.

httpie - A good tool for sending HTTP requests without the complexity that is CURL.

yapf - Code formatting tool made by google. Very useful in a team enviroment to ensure all code looks as if it was written by a single developer.

rauth - Helpful library for people utilizing the OAuth workflow ontop of requests.

ptpython - Best python REPL.

As a software engineer/artist I could spend copious amounts of time refactoring. However, at some point products have to be shiped. You can't just keep pushing deadlines back further and further. Instead I utilize the tips below to provide functional code that is also clean.

The DRY Principle.

DRY stands for "don't repeat yourself" and you shouldn't. If you do, STOP! Repeating yourself will never be a good pratice. The programming gods invented functions for a reason. Start now by splitting up your big main() funciton.

Design Patterns

What you might not of know is that there are three major categories of design patterns; creational, structural, and behavioral. Each category having a plethra of tools to help solve everyday software problems.

Understand The Computer

Knowing that the windows registry exists and how to edit it is not "understanding" the inner workings of a computer. Understanding a computer means having knowledge about things such as the datapath on the CPU and different levels of cache.

If you don't understand how a CPU does computations such as multiplication down at a binary level, how are you suppose to think in terms of preformance? When your application is eating up precious cycles for useless tasks, being able to optimize the program and utilizing your compilers internals to elevate things to a kernel level commands can become of utmost importance.

Tools

This one is really just based on how much experience a person has with the utilities of their trade. Currently I use VS Code + a Vim Shortcut plugins. I then utilize flake8 to keep my code style up to par and pylint to ensure no bugs exist in my code.

I'm not saying switch to my tools, I'm saying start diving deep into how they work. Ask yourself, are there any menu's or options you haven't explored before? Mess around with them until you figure out. Can you use a tool in a way it wasn't meant to? Try it. Don't settle, expand it and put it into your pipeline.

Readability

This one is very important. Instead of writing a list comprehension that goes way past the end of the 79 character PEP8 line length, use multiple lines. Name variables approprately and keep packages clean. All things that you should be actively striving for when writing code or refactoring.

Code Reviews

Open source or not you should strive to get someone to review your code. This helps promote a more open enviroment and prevents bugs from making it into production code.

Testing

I could talk for hours on this one. Testing is essential. Start out with unittests and verify small components of the system. Once you get a hold of how to do that, start adding integration tests for business essential parts of your application.

Have a ton of services? Test the main functionality of these programs by running smoke tests. Smoke tests are very light weight scripts that verify the conditions of your system. A good example of smoke tests is ensuring your FTP server is online and that the user can login.

Conclusion

Wrapping things up, you should always strive for clean and readable code. I hope that some of the items I've listed above can enable you and your team to create amazing and beautiful products for people all over the world.

Please feel free to share this post and comment anything I left out or might be able to improve on for my own sake.

Update #1:

Today is the day I finally ordered my raspberry pi 3 model b, powersupply, and a 32GB SD card. I had an ich to get a bigger card, but then I came across this which led me to believe a 32GB card might be it's max and I don't want to pay more money only to find out it doesn't work.

List Of Items Purchased:

$36.00 - Raspberry Pi 3 Model B

$15.10 - SanDisk Extreme 32GB microSDHC

$9.99 - CanaKit 5V 2.5A Raspberry Pi 3 Power Supply

These three items tallied up to an exact: $61.09(including sales tax)

I'll have to get a case at another point and time. The one's on amazon wern't very appealing.

Original Article:

Last week, while I was on vacation, I got inspired by my Uncle to build an arcade emulator. Then, coming back to work I found a hacker news post about an article from coding horror detailing some of the specifics about raspberry pi based emulators. I figure the first step to see if I want to take on this project is to price out all of the hardware. Then depending on how much disposable income I can afford, I'll hopefully act upon this impulse thought.

Enjoy! :)

Raspberry Pi Kits:

$49.99 - Raspberry Pi 3 Starter Kit

Raspberry Pi:

$35.00 - Raspberry Pi 3 Model B

$36.00 - Raspberry Pi 3 Model B

Power Supplies

$9.99 - CanaKit 5V 2.5A Raspberry Pi 3 Power Supply

Micro SD Cards:

$15.10 - SanDisk Extreme 32GB microSDHC

$12.99 - Samsung 32GB EVO Plus Class 10 Micro SDHC

Controllers:

$7.99 - SNES USB Controller

I've some xbox one controllers I can probably use.

Just run the command below to download and run a script from the docker website. It will figureout which install is approprate for your platform and everything! Isn't it sweet?

wget -gO- https://get.docker.com/ | sh

Enjoy :)

I am new to pelican, so in reality this is just the steps I used to setup my pelican blog. Feel free to experiment and do things differently.

Create your virtual enviroment.

Anytime you are using python, you should always utilize a virtual enviroment. In this tutorial we will use virtualenv. This will prevent us from junking up our global install of python.

For first timers I suggest visiting the hitchhiker's guide to python, as they have a very comprehensive tutorial on getting started with virtual enviroments.

Install Pelican

Verify that your virtual enviroment is active so that we can start to install packages. Here we will install pelican and it's helper package Markdown.

pip install pelican Markdown

Pelican Quick Start

Now it's time to setup the site. Run the quick-start command and enter in details such as the site name and description.

pelican-quickstart

Note: Make sure you decided on a catchy name!

Pick a Theme

Pelican comes with a default theme, which I personally dislike. It just doesn't capture my personality very well. Luckily there is a site called pelican themes that will give us a good selection of 3rd party themes we can use on our site.

I personally really like the flex theme. Clone the theme of your choice and prep for your first blog post. Make sure you setup your pelicanconf.py file as deamed via the README in the themes repo.

Create your first post

Write your first post in markdown and place it inside the content folder.

Compile your blog.

Time to have pelican generate your static HTML blog for you! Just run the command below.

pelican

Simply upload your output folder and enjoy!

Hello internet!

I'm just going to get stright to the point, I use to have a personal/tech blog @ tech-fyi.net but like bread, it got stale. I started it over 7 years ago when I was in high school. Since then, I have graduated with a bachelor's degree in Computer Science from Purdue and have moved on to more interesting things in life.

More soon. Don't forget to interact with me on Twitter!