jump to navigation

Expose the Internals April 29, 2016

Posted by PythonGuy in Uncategorized.
add a comment

I think it would be cool if Python exposed some of its inward functionality. These are some of the bits I would like to have:

  • The function that maps function arguments to parameters. It would take a function parameter list, an argument list and an argument dict and convert it to a dict of variable-value mappings.
  • The function that creates functions. I know I can use exec for this, but I think it would be preferable to allow me to specify a string and get back a chunk of un-executed code.
  • The lexer and parser. Along with this, I’d like to be able to create arbitrary data structures representing the output of the parser and put that into exec.

I don’t know how much work it would be to expose these parts. I should look into making a module that does this. It can’t be that hard.


More on my ideal language April 29, 2016

Posted by PythonGuy in Uncategorized.

Consider the following possibility:

  1. All statements and expressions are not evaluated by default. That is, “1+1” does not mean “2”, but literally, “add 1 and 1.” When I set “a = 1 + 1”, I am really setting “a” to be “add 1 and 1”. Or rather, it literally means “set a to 1+1”. It doesn’t actually do anything.
  2. There is a language feature to execute statements. I think they use “`” in Lisp. I would rather have it a function just like any other, though. I’d call it “exec”. So, “a = 1+1; exec(a)” would give me 2. But do you see the problem? “exec(a)” literally means “exec(a)”, which does not execute it but describes the action of executing it. So perhaps “exec” is special in that it is the only function that actually executes anything.
  3. With such a feature in place, it is simple to see how to make the if statement into an if function. IE, “if(x<y, return(7), return(5))”. This function would take three parameters, none of them executed until if() said so. It would execute the first parameter, and then choose between executing the second or third.

I think the above features would give me the equivalent of Lisp’s macros in a Python-like syntax. It would be interesting to try and program completely in such a manner, though. I would like to invent a language just to see what it would be like.

Verbs don’t need Nouns April 29, 2016

Posted by PythonGuy in Uncategorized.

A shower thought, more or less, about verbs, nouns, and the nature of computer programming languages.

Verbs can live on their own. They don’t need nouns. We see this because in many languages that require nouns, we use placeholder nouns. IE, “It is raining.” What is “it”? It is literally nothing. We are describing the state of the universe. We could’ve dropped the subject altogether and lost no meaning at all: “Is raining.” That’s almost perfectly clear, even in a language like English where every sentence needs a subject noun.

Nouns, however, cannot work alone. They need a verb to tie it into the universe somehow. For instance, “Baby!” means, “There is a baby!” The verb, “is”, is implied, and is part of the idea being expressed. Or, in a auto repair shop, “7 mm” would mean, “Please hand me the 7 mm wrench.”

I think you’ll find that as you look at the fundamental ideas in language, verbs will always be the most important. Nouns alone can only express phrases or parts of ideas, not complete, whole, independent ideas. Sentences are the basic building blocks of any language, and every sentence needs a verb, even if it is the lowly connecting verb or the lowly declaration-of-existence verb.

In programming languages, we write different kinds of statements. These statements generally take one of the following form:

  1. Declarations. There is this thing called X and this is what it is like. In Python, class and function declarations fall into this category.
  2. Procedure descriptions. The fundamental unit here is a single unit of action. Assign this value to that variable, call this function. All procedure descriptions can be broken down into more basic procedure descriptions, all the way down to the fundamental operations the language provides. In this way, the operators and basic functions form the core of the language that all the rest of the language relies on.

Honestly, it’s very hard to distinguish between the above two types of statements, suggesting there may be some more fundamental concept. After all, a class statement is really saying, “Create a class object given these parameters and assign it to that variable which happens to be the same as the class name.” The same goes for function declarations.

In the end, I believe the fundamental unit of any language, even Java, is the function. All languages are really a subset of Lisp or Scheme.

Viewing languages this way is certainly liberating. It’s hard to explain why, but when you see that all things are really the same thing done in different ways, your mind has an easier time inventing new things.

Why I Don”t Like REST (The Shorter Version) April 22, 2016

Posted by PythonGuy in REST, Uncategorized.
add a comment

I posted earlier why I don’t like REST and avoid it at all costs. I’ve been trying to find a way to summarize it in a succinct idea, or find the one thing that everyone would agree is a “killer” feature, in that it would “kill” the idea of REST altogether.

I think I have it.

REST is bad because REST does not map to any language well. Or rather, any RPC would map much better to any language.

That is, you always have to write boiler-plate code on top of REST just to make it useful in your language. Whereas, with an RPC, boiler-plate code can be auto-generated for any language.

There are 2 reasons why this is:

One, REST only supports 8 operations. They are:

  • GET a list of stuff.
  • POST a list of stuff, replacing the list.
  • PUT a list of stuff, adding to the list.
  • DELETE an entire list of stuff.
  • GET a single item.
  • POST a single item, creating it brand new.
  • PUT a single item, replacing it.
  • DELETE a single item.

If you want more than these 8 operations, or if you want to remove some of these operations (and you typically do), then you’ll need to extend the REST API to handle additional things. There is no simple definition on how these extensions would even work, but that’s not the issue. The issue is that REST as an API is either too much or too little, and rarely, just exactly enough.

Two, all languages are function-based, or rather, procedure based. Every language has at its core something to process individual commands, and something to describe how to execute or run commands with further commands. These commands, which are called procedures or functions, are the fundamental unit of programming. It only makes sense that any layer between two programs should also be defined in terms of functions.

RPCs don’t need any operations by default. Some may have some default operations that define or characterize the service, or maybe some directory operation that tells the client what operations are available and what their parameters are.

To summarize, consider the fellow who has to write the API client code. He is going to put all the code into a module or package of some sort. He is going to introduce a new data structure into the language that represents the connection to the service, along with whatever authentication tokens are necessary.

Then he is going to define the operations that the service allows. He has to give them some sort of name to abstract away the RESTful nature of the service, or else he has to have a leaky abstraction that bubbles up features of the REST service to the ultimate end user of the client library. He has to pick and choose which to keep and which to not implement, and then decide what to do if someone tries to do one of the missing operations. It’s all a terrible mess when using REST.

RPC, on the other hand, is very clear cut. You give your methods the same name as the RPC methods, using the same parameters, adapted to the language of course. It’s so trivial to write a client library that any programmer writing one would be thinking of ways to automate the process using a template. Indeed, most languages would probably tolerate a data-driven architecture for defining the client API.

Another Ideal Language Feature: Dynamic Variables April 12, 2016

Posted by PythonGuy in Uncategorized.
add a comment

Let me add this to the list of ideal language features: dynamic variable scope.

To understand dynamic variable scope, you have to first understand global and lexical scope.

Global variables are accessible anywhere in the program. The problem with global scope is that changes to these variables affect code far, far away from the code manipulating the variable. Also, there is limited namespace at the global level, but this is manageable with nested namespaces.

Lexical variables are accessible only from the function where they are defined. The variables cannot be accessed from outside functions or outside code. This is nice because it avoids the problems of namespace collision in global namespaces, as well as the difficulty in determining how changing a variable could affect other code.

Global variable and lexical variable solve probably 90% of the issues you’ll run into. But there is another case that arises often enough it deserves its own solution. The solution is dynamic variables.

With dynamic variables, the variable is set, but only for functions called from the scope at which it was set. That is, if there are 2 functions, a and b, and both refer to the variable c, if b is called from a, then a can set a value for c that b will see. After a completes, that variable no longer exists.

To further define it, let’s look at the 4 things you can do with variables.

  1. Declare the variable. For dynamic variables, you declare the variable at a point in the calling stack. Functions called from below that stack can see it, but the variable is released when that level is left.
  2. Assign a value. For dynamic variables, any function called from that level in the stack can assign a new value to it, and it will apply to the entire stack.
  3. Retrieve the value. For dynamic variables, retrieving a dynamic variables will retrieve the variable from the stack level it was defined in.
  4. Delete the variable. This would retire the variable for all levels in the stack. It is automatically released when the stack level is returned from, just like for lexical variables.

Why would you want to do this? In today’s world, there are two obvious use cases. Understanding these use cases will help you see their value.

First, consider what happens in a web app. For each request that comes in, certain values are set to certain variables. These values should be accessible to all the functions called from the request handlers, and the variables called from there, but we shouldn’t get cross-call pollution. That is, if you set the HTTP request headers to a global variable, then operated the server in an asynchronous fashion, all the functions would only see the latest request, not the request that those functions are working on. Today, people use thread-level scoping, a sort of global scope but for an entire thread. This has problems because it means you have to use some sort of thread mechanism which may not be suitable.

Second, consider how we handle configuration. The program starts, reads in configuration, and sets global variables to correspond to the configuration. The entire program can then access this configuration. What happens if you temporarily want to override configuration for a certain section of code? Like, let’s say we want to turn the log level up to DEBUG, but only when performing a certain action. We can do this lexically, but really, we sometimes want to do it for the dynamic scope: this function and all the functions called from it. Really, configuration belongs in a dynamic variable, silently passed to all functions called from the main() function that accepts the configuration.

Unfortunately, dynamic variables are an advanced concept that requires a thorough understanding of the calling stack and are only useful for advanced programming techniques that beginners rarely encounter. Thus, their need isn’t as obvious as global and lexical variables, and so many programming languages don’t even allow them. However, to advanced programmers, they can be extremely useful.

The good news is that in programming languages that allow dynamic variables, they are rarely used, so beginners won’t encounter them. And when they do, there can always be a little explanatory note since it is a rare feature.

Python Daemons April 12, 2016

Posted by PythonGuy in Uncategorized.
add a comment

I’ve done some experiments to make true daemons in Python.

The basic concept is to write a wrapper that will launch a daemon with a function call. The function call could call os.exec which could run anything.

Basically, use the daemon module from PyPI.

If that seems unpalatable, there is the possibility of using systemd as a user, but that’s only available on systems with systemd.

A Function that Creates a Function April 4, 2016

Posted by PythonGuy in Uncategorized.
1 comment so far

I was trying to write a neat little module that would generate REST API clients given just a few bits of information. I got stuck at trying to generate functions given data.

While I could use a closure, I can’t easily control the function signature. What I really wanted was a function that would take the parts that make up a function into an actual Python function, much like the way metaclasses work. In particular, I wanted to take advantage of Python’s native function signature handling code, and not try to re-implement¬† it myself.

I poked around a bit but the best example I could find was decorator.py’s decorator function. This function creates a new function that has the same signature as another function. It actually cheats, though, relying on Python’s ‘exec’ statement to run a string as code.

It turns out that Python has not function that generates functions. It seems like it would be even more useful than type(), which can generate classes.

So I’m left to using exec. Sub-optimal, but whatever.

My next favorite programming language after Python will have a function that generates functions. Rather than passing a string in as the body, you could specify a function to call. So really, it will just allow you to specify the signature of a function.

Or maybe I’ll just ditch the idea of function signatures altogether. Every function takes any set of parameters.

By the way, functools.partial has to do the same thing, but as near as I can tell, it cheats by ignoring the signature.