jump to navigation

Why You Should Never Do Multithreaded Programming November 3, 2016

Posted by PythonGuy in Uncategorized.
add a comment

TL;DR: As your software gets popular, you’ll want to scale your program. The temptation is to use multi-threaded programming techniques, but this is a very complicated paradigm that has bugs that are vicious and very difficult to find and remove. As your software grows more, you’ll have to adopt the multi-process paradigm. So just skip multi-threaded programming and go to multi-process programming. If you need multiple threads, use greenlets instead.

I was first introduced to multi-threaded programming sometime in the 90s. I recall reading about it in C/C++ programming books. It was wonderfully complicated, just the sort of thing to excite my teenage mind. As I started using it, however, I soon realized that what the authors of the book said about it wasn’t a joke. Deadlock and other issues tarnished the image of multi-threaded programming.

But what was I to do? I had a single computer running a single core, and if I wanted to do work really fast, I had to use multi-threaded programming techniques.

Over time, I learned about the asynchronous programming model. In this model, you don’t have real threads. Instead, you have a central function that dispatches to various other functions which do a little bit of work on a task and return control to the central function. If the central function is built around a select() or poll() function, and they each processed a file handle, you could handle a very large number of simultaneous connections with a single thread. Provided that there wasn’t a lot of data flowing and there wasn’t a lot of work to be done, you could get tremendous throughput compared to multi-threaded programming.

I read about how Google was using Python a few years after that. They would write simple, single-threaded applications. Then when they needed the apps to do more, they would just start up more machines and run the app on those machines, and then use load-balancing techniques to direct the workload to each machine. Programming in this style is called multi-process programming, what we’ve been doing all along since Windows 95 introduced us modern operating systems. The difference was that you couldn’t share memory. You had to open pipes over the network to communicate. That was your only option.

Of course, if the two processes are running on the same machine, and you match the number of processes to the number of cores, it’s like having several little virtual machines inside of a single machine. As long as you only communicated with pipes, you could see how to take these processes and move them to other services. Network latency was higher than local UNIX socket latency, but that could be dealt with.

In short, multi-threaded programming is simply unnecessary. There is no reason to use it. If you feel tempted to do it, just use multi-process programming knowing that your additional processes can be moved to another machine.

A lot of work has been done figuring out how to make one process do more work. Before I continue, let me explain why this work doesn’t really matter.

When you have an exponentially growing demand for your program, like, for instance, it is doubling every 6 months or something like that, then you’ll need to scale your operations as well. If it takes 10 machines to handle 10,000 units, how many machines to handle 20,000 units? The answer is simple: About 20. And 40,000 units? Now you need 40 machines. Each time you double, you double the number of machines.

Let’s say I could fix the code so it ran twice as fast, or rather, needed half as many compute resources. All I’ve done is bought some time. See, when you’re handling 20,000 units with 20 machines, I’m handling the same with 10 machines. 6 months from now, I’ll be at 20, and you’ll be at 40. Another 6 months and we’ll both double again. All I’ve done is said I’m going to hold off on purchasing more resources for 6 months.

When you look at things this way, then the amount of time you buy by making your code more efficient works out to be a linear unit of time with diminishing returns. Make my program 2x as good, and I buy 6 months. 4x as good gives me only 6 more months. 8x as good gives me only 6 more months. And 16x as good buys me a year.

If it takes me 6 months to make my code 2x as good, I’m wasting my time. I’d be better off writing a new program and buying more machines.

That said, some people aren’t running cash cows and do care about how much their services cost because they didn’t do a good job estimating how much profit there would be. They say things to their workers like, “We can’t afford to buy twice as many machines”. What changed is not the code quality, but the profit margin. They may have been making $1 per unit at 10,000 units, but at 20,000 units, they are only making $0.50. And so what they’re really saying is, “We have to stop growing as a company.” If your company leaders are saying that, it’s time to find a new job.

Of course, sometimes they say things like, “We can’t afford to buy those machines today (because our credit limit won’t allow it or we don’t have investors), but we will have that money tomorrow (because we know it is worth it.)” When they say things like this, then and only then should you waste time making your program run faster.

All the above said, we have learned some seriously awesome tricks to making programs work faster. It’s called asynchronous programming, and it allows a single process to behave like 10,000 processes. That’s 2x applied to itself 13 times. So if you’re growing at a rate of 2x every 6 months, that will buy you 6 or 7 years, which is longer than I’ve lasted at pretty much every job I’ve ever worked at. That’s like several lifecycles of technologies on the internet. In Python, the easiest way to manage asynchronous programming is with greenlets. (Avoid twisted. Seriously.)

So do yourself a favor, learn about greenlets, and learn how you can program in a synchronous style asynchronously.

But don’t bother learning about multi-threaded programming. Just know that you never, ever want to go down that road.

Advertisements

Makefile November 3, 2016

Posted by PythonGuy in Uncategorized.
add a comment

Now that I am older, I see things that the younger programmers don’t see. Among those things I see are people not using software for the wrong reasons. Just because something is old and complicated doesn’t mean it isn’t the right tool for you.

The ancient and venerated Makefile is, in my mind, the most wonderful tool you can use to make your software development smoother and simpler. Ignore it at your peril. Avoid it at your detriment.

You know how you tell your co-workers about how to do stuff? Rather than tell them, just put it into a Makefile. That way, they don’t have to remember anything but “make <the thing you want to do>”. And if they forget what they can do, they can just pop open the Makefile and see what the various targets are and how they work.

The Makefile syntax is cryptic and is very uninviting. Any sufficiently useful Makefile will have commands that are difficult to decipher. However, with a little practice, and a few minutes with the documentation on GNU make, you too can be an expert. The best part is you know that the Makefile syntax isn’t going to change anytime soon. What you learn about it today will probably be useful 30 years from now.

I won’t bother explaining how make works. There are several similar projects out there, but none of them compare to the power of the Makefile. Why do these other programs exist when Makefile is so much older? I believe it is because of a few reasons.

  1. People didn’t take the time to learn make. I can’t blame them, but really, there is a reason why it is so complicated. As you learn to use make, you will grow to appreciate that complexity.
  2. Vendor lock-in. If the vendor can get you to depend on their make clone, then you’ll never go back to using the free and open source make system. Once they’ve got you in their web, you’re not going anywhere except to the land of regret and sorrows.
  3. People don’t believe make should be so complicated. They come up with their own solutions that end up being even more complicated.

The philosophy behind make is rather simple.

  • You have targets, usually files you want to build. However, targets can be “virtual”, meaning that once the target is complete, there is no file leftover.
  • Targets depend on requirements. These requirements are also targets. Ultimately, the requirements boils down to a set of files or imaginary virtual targets.
  • You have recipes that list the commands needed to build the target from the requirements.
  • You have a ton of configuration parameters that will allow you to create a flexible Makefile that is truly cross-compatible against a wide variety of platforms. These can be a rather difficult chore to get right, so typically there is the infamous “configure” script that will figure out where you keep everything on your system and make sure you have the right things installed and they are the right versions.

When you use make, you can depend on things like Ruby’s gems or Python’s pip system. You can literally put make on top of anything, since all it does is call programs with parameters. For instance, I’ve seen Makefiles in Java projects with Ant for a configuration system.

Make is a lingua franca. Or, in more modern terms, it is the English Language of building computer software. It is so universally adopted that it becomes the common way to express how to build things.

Do yourself and your co-workers and co-collaborators a favor. Write a Makefile today. Start using it. Start learning how they work. Study other people’s Makefiles. This is not wasted effort.