OpenJDK Porting Experiences:
The Good, the Bad, and the Downright Ugly

Steve Poole (IBM) & Volker Simonis (SAP)

Who we are - the PowerPC/AIX Port

The PowerPC/AIX Port Project: http://openjdk.java.net/projects/ppc-aix-port

Jointly initiated (May 2012) and currently driven by IBM and SAP
Provides a full-featured version of OpenJDK on Linux/PowerPC and AIX/PowerPC

JEP 175 : PowerPC/AIX Port
OpenJDK Wiki: https://wiki.openjdk.java.net/display/PPCAIXPort
Mailing list: http://mail.openjdk.java.net/mailman/listinfo/ppc-aix-port-dev

Currently three forests:

State of the PowerPC/AIX Port

C++-Interpreter and C2-Server compiler are fully functional in JDK7 and JDK8
Successfully passed the Java SE 7 TCK on Linux and AIX in 64-bit mode
Regularly updated with upstream repositories (currently jdk7u40 and jdk8 b106)
JEP 175 : PowerPC/AIX Port endorsed with:
- detailed PowerPC/AIX Port Integration Plan
Staging repository with reviewed changes ready for upstream integration

C++-Interpreter fully functional on both Linux and AIX
Class library fully functional on Linux and basic version on AIX

Nightly builds and tests on porting platforms and Oracle platforms

Build Results (1)

http://cr.openjdk.java.net/~simonis/ppc-aix-port/

OpenJDK PowerPC Port for Linux and AIX Nightly Build Results

Build Results (2)

http://cr.openjdk.java.net/~simonis/ppc-aix-port/

Test Results

http://cr.openjdk.java.net/~simonis/ppc-aix-port/test/test_results.html

Performance - JVM98

Performance - JVM2008

Our background - the SAP JVM

The SAP JVM supports Java 1.4, 5, 6, 7 and runs on 15 platforms:

Linux on x86, x86_64, IA64, PPC64, zSeries; Windows on x86, x86_64, IA64;
Solaris on x86_64, SPARC; HPUX on IA64, PARISC; AIX and AS400 on PPC64; MacOS X on x86_64

..and we provide support for any SAP JVM version until the end of days:)

The SAP JVM is derived from the Sun/Oracle code base:

with custom ports to the platforms not supported by Oracle
with enhancements mainly in the supportability area
with special ad-dons (e.g. SAP JVM Profiler)
See What's Cool in the SAP JVM [CON7884] (Thursday, Sep 26, 15:30 - 16:30 - Hilton - Imperial Ballroom B )

We constantly integrate Oracle changes:

leading to an increasing code divergence between our and Oracle's version
initially we only got "source drops" from Sun/Oracle:
- i.e. the source code of every released Java version and every JDK update
after more than 7 years, merging has become a nightmare

Our background - IBM J9

How we joined the OpenJDK

Announced at JavaOne 2006

Open Source implementation of Java SE.
Licensed under GPLv2 (with Classpath exception).

SAP can't use OpenJDK directly:

Because its customers expect a commercially licensed JDK.
Because it also has to deliver JDKs 1.4 and 5.

It took 5 years until SAP "officially" joined the OpenJDK project:

Convincing SAP executives/developers to join an open source project was not easy.
Oracle's Sun acquisition was not helpful either:)
We had to ensure that we get contributed code back under our commercial license.

Today, the OpenJDK is a playground and collaboration space for different implementers:

IBM, RedHat, Apple, Twitter, Azul, SAP, ..

The OpenJDK Source Tree

The OpenJDK consists of two major building blocks.

The HotSpot Virtual Machine (~1.600 files, ~340.000 loc)

coded in C++ and Assembler

well organized by operating system and architecture

hotspot/src/cpu       hotspot/src/cpu/ppc      hotspot/src/os/aix        hotspot/src/os_cpu/aix_ppc
hotspot/src/os        hotspot/src/cpu/sparc    hotspot/src/os/bsd        hotspot/src/os_cpu/bsd_x86
hotspot/src/os_cpu    hotspot/src/cpu/x86      hotspot/src/os/linux      hotspot/src/os_cpu/bsd_zero
hotspot/src/share     hotspot/src/cpu/zero     hotspot/src/os/posix      hotspot/src/os_cpu/linux_ppc
                                               hotspot/src/os/solaris    hotspot/src/os_cpu/linux_sparc
                                               hotspot/src/os/windows    hotspot/src/os_cpu/linux_x86
                                                                         hotspot/src/os_cpu/linux_zero
                                                                         hotspot/src/os_cpu/solaris_sparc
                                                                         hotspot/src/os_cpu/solaris_x86
                                                                         hotspot/src/os_cpu/windows_x86

The Java class library (~11.000 files, ~235.000 loc)

coded in Java with a considerable amount of native parts(C and C++)

the platform-dependent parts only divided into a *nix flavor, Windows and recently MacOS

jdk/src/share                     (~ 7800 Java and 650 C files)
jdk/src/macosx                    (~  370 Java and  10 C files)
jdk/src/solaris                   (~  350 Java and 270 C files)
jdk/src/windows                   (~  190 Java and 190 C files)

The HotSpot VM

The HotSpot VM first appeared in 2000 with Java 1.3 and is constantly evolving since then.

Mostly architecture dependent parts:
- Java Bytecode Interpreters
  - Template Interpreter
  - C++ Interpreter
- JIT Compilers
  - C1 aka "Client compiler"
  - C2 aka "Server compiler"
Mostly OS dependent parts:
- Runtime system
  - Memory handling (VM Heap, Java Heap, CodeCache)
  - Process/Thread/Signal handling
Mostly generic parts:
- Garbage collectors
- Class loader/verifiers

Porting the HotSpot VM - Effort

Taking the Linux/x86_64 version as reference implementation:

hotspot/src/share            (~1100 files, ~100.000 loc)

hotspot/src/os/linux         (  ~25 files,   ~9.000 loc)
hotspot/src/os_cpu/linux_x86 (  ~20 files,   ~3.500 loc)
hotspot/src/cpu/x86          ( ~100 files,  ~90.000 loc)

these numbers include both interpreters and both JIT compilers

We started with an interpreter-only version (using the C++Interpreter on Linux):
```
hotspot/src/os_cpu/linux_ppc (   11 files,   ~1.500 loc)
hotspot/src/cpu/ppc          (   61 files,  ~23.000 loc)
```
- successfully runs JVM98 and can be used as bootstrap JDK

C2 JIT compiler:

hotspot/src/os_cpu/linux_ppc (+   1 files,+    ~400 loc)
hotspot/src/cpu/ppc          (+   7 files,+ ~15.000 loc)

AIX port:

hotspot/src/os_cpu/aix_ppc   (  ~13 files,+   ~2000 loc)
hotspot/src/os/aix           (  ~26 files,+ ~15.000 loc)

The C++Interpreter

Consists of a huge interpreter loop written in C++,
and a so called "frame manager" written in Assembler.
- The "frame manager" is a frameless method which handles Java method invocations.
- This keeps the Java frames continuous on the mixed Java/Native stack.
- There's only one activation of the C++ interpreter loop on top of the stack.
- See hotspot/src/cpu/ppc64/vm/cppInterpreter_ppc64.cpp (~3000 lines of assembler code)

+------------------+
|                  |  C++ interpreter loop
+------------------+
|xxxxxxxxxxxxxxxxxx|  java frame n
+------------------+
:       ....       :
+------------------+
|xxxxxxxxxxxxxxxxxx|  java frame 0
+------------------+
|//////////////////|  vm
|//////////////////|

The MacroAssembler for your CPU is required for the Interpreter port!

The OpenJDK contains macro assemblers for x86 (~12.000 loc), SPARC (~8.000 loc) and ppc64 (~9.000 loc).
- see hotspot/src/cpu/<arch>/vm/assembler_<arch>.{hpp,inline.hpp,cpp}
The Macro Assembler can be reused for the JIT compilers.

The C2 "Server" JIT Compiler

The C2 "Server" JIT Compiler is the biggest (and most complicated) part of the HotSpot VM.
It consists of three main parts:

the generic optimizer written in C++ (under src/share/vm/opto)
an "Architecture Definition Language" and Compiler written in C++ (under src/share/vm/adlc)

the "Architecture Definition" file written in ADL (under src/cpu/<arch>/vm/<arch>.ad)

hotspot/src/share/vm/opto         ( ~110 files, ~128.000 loc)
hotspot/src/share/vm/adlc         (  ~23 files,  ~26.000 loc)

hotspot/src/cpu/x86/vm/x86_32.ad  (              ~14.000 loc)
hotspot/src/cpu/x86/vm/x86_64.ad  (              ~13.000 loc)
hotspot/src/cpu/sparc/vm/sparc.ad (              ~10.000 loc)
hotspot/src/cpu/ia64/vm/ia64.ad   (              ~26.000 loc)
hotspot/src/cpu/ppc/vm/ppc_64.ad  (              ~14.000 loc)

For every new architecture the corresponding AD file has to be written which means:

defining the different registers
defining the different calling conventions
defining concrete "encodings" (i.e. assembler instructions) for the abstract optimizer nodes

Changes in shared code

Basic changes:

configure/make infrastructure (the new build System is of great help here!).
"Include cascades" of platform headers where needed in shared code
- This was easier with the old "IncludeDB" but nevertheless we don't want it back :)
C/C++ syntax adaptions for compilers on PPC (e.g. for IBM's xlc)
- Inline assembler, pragmas, precompiled header support, Platform dependent macros
safefetch: use generated code instead of inline assembly: better portable
around 15 memory ordering fixes in runtime code (e.g. TaskQueue)

C++Interpreter changes:

bytecode profiling (large)
weak memory ordering (sophisticated)
on stack replacement (OSR)
support for G1 garbage collection
method handles
compressed Oops

Porting the Class Library - Effort

Shared Java classes (no porting effort)

corba/
jaxp/
jaxws/
langtools/
nashorn/
jdk/src/share/classes/                                     Linux/PPC           AIX/PPC

Platform-dependent Java classes

jdk/src/macosx/classes             (~  370 files)
jdk/src/solaris/classes            (~  350 files)                          19 files/1600 LOC
jdk/src/windows/classes            (~  190 files)

Platform-dependent C/C++ code

jdk/src/macosx/native              (~   10 files)
jdk/src/solaris/native             (~  270 files)       2 files/10 LOC     26 files/1500 LOC
jdk/src/windows/native             (~  190 files)

Build infrastructure

common/                            (~   60 files)       7 files/500 LOC    10 files/1200 LOC
jdk/makefiles                      (~  130 files)       8 files/350 LOC     8 files/2700 LOC

Porting-Lessons learned

During the last years we've ported the JDK to several platforms and we learned that:

The HotSpot has a very steep learning curve.
There is not much documentation available.
The HotSpot is a fast moving target.
The class library is "not designed" for new platforms.

OpenJDK shortcomings (1)

The OpenJDK is still dominated by Oracle

Still very few "external" committers/reviewers.
All the project infrastructure is owned by Oracle.
Still dependencies on closed sources (e.g. build system or HotSpot).
External committers can not push to some projects (e.g. HotSpot).
Sources for security fixes only after Oracle release their proprietary builds.

The OpenJDK infrastructure

No public Wiki/Bug-Tracker
No modern code review system.
No good test infrastructure (i.e. the Java TCK not generally available).
No binaries (i.e. no regular OpenJDK builds).
Mercurial repositories and Webrev servers only accessible through SSH.

OpenJDK shortcomings (2)

There's no general procedure for the integration of bigger contributions.

The Java Community Process (JCP) is too heavy-weight.
The JDK Enhancement Proposals (JEP) process is dominated by Oracle
- Requires endorsement and funding by Group/Area leads.
- The OpenJDK Lead (appointed by Oracle) ultimately decides which JEPs to accept for inclusion into the Roadmap.
Required quality standards can not be met by "external" contributors.
- There exist no freely accessible tests and success metrics

Questions?

Rechenmaschiene von Philipp Matthäus Hahn (1739-1790) Quelle: Württembergisches Landesmuseum Stuttgart

OpenJDK Porting Experiences: The Good, the Bad, and the Downright Ugly