Can We Trust Dynamic Optimization

Dynamic Compiling, Dynamic Optimization and Code Morphing are some of the buzz words you’ll find used often in programming circles especially online.  Yet for anyone who’s released a piece of code or an update that’s gone wrong, they are phrases that can make you very nervous.   So what do these phrases actually mean and what is their impact on today’s technology?

Dynamic Compiling and Dynamic Optimization are the most common names for it. “Code Morphing”, the most hip term so far, was coined by David Ditzel of Transmeta. The Tao Group prefers the name Dynamic Binding. However, there are still people who use another name altogether: Binary Translation (because translation is such an essential aspect of the technology). But don’t worry. I will use the nice simple abbreviation CF (Code on the Fly) to prevent ourselves from getting all tongue-twisted.

Yet our technology is everywhere now and in some senses it’s impossible to keep up with all the improvements and updates that are necessary.   Even applying security patches to code can be a huge job and many organisations struggle to cope with even this basic requirements.  Imagine the thousands of servers and proxies in most corporate network which need constant updates without even considering normal efficiency based code changes.  This becomes even more important when you have critical roles for specific servers although the volume is obviously reduced.  The ‘point of entry’ server for large backconnect databases you find for people trying to access bulk shared proxies such as these potentially affect thousands of individual connections.

Like most technologies, CF is really a simple concept disguised by big fancy words. An example is the best way of understanding what it’s all about.

We will use Java as our example because it is so popular these days. Java programs are really “pretend-programs” written for a “pretend-computer” called the Java Virtual Machine (JVM). To run a Java program, a “real-program” is needed that pretends to be the JVM by emulating it. Now emulation is very slow. So to speed things up a Just-In-Time (JIT) compiler is used. The JIT compiler converts the Java program into a “real-program” by generating code for the “real-computer” while the JVM program is running (or in other words, on the fly). JIT compilers are faster than emulators because they execute Java programs more directly than the emulator’s slow interpretive layer. A JIT compiler is an example of Dynamic Compiling. Pretty straight forward, uh? Traditional compilers don’t fall into the CF category because they generate code before the software is delivered.

Even microprocessors use Dynamic Compiling. Instruction level parallelism (ILP) (also known as Out-Of-Order [OOO] Execution) is attained by re-arranging instructions, which is essentially a form of Dynamic Compiling. The entire Pentium line translates the horribly complex x86 opcodes into the micro-ops actually understood by their underlying RISC engines. Similarly, Transmeta’s Crusoe and the Elbrus 2000 translate x86 and IA-64 instructions into those actually used by their respective VLIW processors.

There is a whole spectrum of opportunities available for optimization. Static optimization at compile time is one point on this spectrum. Dynamic optimization at runtime is another. Dynamic Optimization is a more advanced form of CF. It aims to improve the performance of (already) native programs. This task is done by determining the hottest bits of code and laying them out next to each other. Since the code is no longer spread out all over memory, there is less trashing in the instruction cache. These straight lines of code are then optimized using traditional techniques such as loop unrolling, constant propagation and dead code elimination.

They are particularly useful in the role of e-commerce servers where dynamic updates mean that downtime is minimized.  Many of these sites costs lots of money to update particularly when you consider the volumes of sales involved.  However the upgrades are essential because they are frequently the target of abuse, either criminal or people trying to abuse the systems.  Once example is where people use so called ‘sneaker proxies’, read about them here, to bypass restrictions and create resell markets.

A technique unique to Dynamic Optimization is the removal of each method’s entry and exit code, thus effectively inlining the entire function on the fly. These optimizations address the inability of compilers to optimize across method boundaries, virtual function calls, DLLs and components. Compilers can not see past these boundaries because they don’t know which calls will be made at runtime. They also don’t know what the user input or targets of dynamic linking will be. Thus Dynamic Optimization fills a niche that compilers cannot.

Leave a Reply

Your email address will not be published. Required fields are marked *