Selective setAccessible permissions using Byte Buddy and the Java SecurityManager

The Java SecurityManager provides a lot of good permissions for restricted what untrusted code can access. The downside of the standard permissions is that many of them are too broad, making it hard to enable permissions around commonly used functionality without defeating the purpose of the restricted security. The permissions around reflections are one of these cases, where the permissions around the setAccessible reflections method is either all on or all off. It is possible to work around this default behavior, providing more selective fine-grained permissions, by using bytecode weaving. However, for any of this this to work the Java SecurityManager must be used along with loading the code that these permissions will be applied to from a separate jar, probably using a custom or at least isolated instance of a URLClassLoader. This is a prerequisite which has already been well documented by Will Sargent in his post Self-Protecting Sandbox Using SecurityManager as well as my own example sandbox runtime environment sandbox-runtime.

The main permission in question around the setAccessible method is ReflectPermission("suppressAccessChecks"). When this permission is disabled it disallows changing the accessibility of fields, methods, and constructors. By default the SecurityManager is not enabled, which means all permission checks are disabled. Since this is the standard way Java is run, many common libraries, and even many alternative JVM language are dependent on being able to use reflection to access private fields and methods. If you do need to run code with a SecurityManager, then the ReflectPermission("suppressAccessChecks") permission defeats all security provided by the SecurityManager. Untrusted code can make use of setAccessible to work around and disable the SecurityManager entirely.

It may appear that the only two choices are between having a secure environment and disallowing untrusted code from using libraries or languages that require reflection, or allowing such access by having an unsecure environment. However, there is an alternative if you are willing to put in the work. Libraries like Byte Buddy make it possible to replace methods of existing classes with new implementations, at runtime, through the use of a Java agent.

I have implemented a custom, more fine-grained, permission around access to the setAccessible method. In my examples I have chosen to use the code source or the ClassLoader as the determining factor; code that attempts to call setAccessible will only be granted permission if both itself and the class it is attempting to change the accessibility of are from the same class loader. In other words, this reflections method can only be used on classes from the same jar as the untrusted code; it cannot be used to access private fields of standard Java classes. This is just what I chose, as it fit best for the security settings I’m targeting. The determining factor could be setup in many different ways, including a package or class whitelist or blacklist.

The first piece that is needed is the new custom Permission. This permission uses the class loader of the caller and callee to determine whether permissions should be granted or not.

public class UserSetAccessiblePermission extends Permission {
private final ClassLoader loader;
public UserSetAccessiblePermission(ClassLoader loader) {
this.loader = loader;
public boolean implies(Permission permission) {
if (!(permission instanceof UserSetAccessiblePermission)) {
return false;
UserSetAccessiblePermission that = (UserSetAccessiblePermission) permission;
return that.loader == this.loader;
// equals and hashCode omitted
public String getActions() {
return "";

With the permission class in place, next we need a place to actually check the permission. This will happen in our custom implementation of the setAccessible method that will replace the standard setAccessible method. This can be done using Byte Buddy by creating our custom implementation as a static method with method parameters compatible with the standard setAccessible.

This class has a few interesting parts. It uses the @This annotation from Byte Buddy to get the object the original setAccessible method was called on. Access to this object can be used to retrieve the class loader of the object the accessibility is attempting to be changed on (callee). If the class loader is not an instance of the user class loader, then we just check the standard permission, but if it is a user class loader, then we continue with checking the custom permission. To check the permission we create an instance of the new permission with the class loader of the callee. If the caller has the same permission with the same instance of the user class loader, then the permissions is granted (a SecurityException is not thrown).

public class AccessibleObjectStub {
private final static Permission STANDARD_ACCESS_PERMISSION =
new ReflectPermission("suppressAccessChecks");
public static void setAccessible(@This AccessibleObject ao, boolean flag)
throws SecurityException {
SecurityManager sm = System.getSecurityManager();
if (sm != null) {
Permission permission = STANDARD_ACCESS_PERMISSION;
if (isFromUserLoader(ao)) {
try {
permission = getUserAccessPermission(ao);
} catch (Exception e) {
// Ignore. Use standard permission.
private static Permission getUserAccessPermission(AccessibleObject ao)
throws IllegalAccessException, InvocationTargetException, InstantiationException,
NoSuchMethodException, ClassNotFoundException {
ClassLoader aoClassLoader = getAccessibleObjectLoader(ao);
return new UserSetAccessiblePermission(aoClassLoader);
private static ClassLoader getAccessibleObjectLoader(AccessibleObject ao) {
return AccessController.doPrivileged(new PrivilegedAction<ClassLoader>() {
public ClassLoader run() {
if (ao instanceof Executable) {
return ((Executable) ao).getDeclaringClass().getClassLoader();
} else if (ao instanceof Field) {
return ((Field) ao).getDeclaringClass().getClassLoader();
throw new IllegalStateException("Unknown AccessibleObject type: " + ao.getClass());
private static boolean isFromUserLoader(AccessibleObject ao) {
ClassLoader loader = getAccessibleObjectLoader(ao);
if (loader == null) {
return false;
return UserClassLoaders.isUserClassLoader(loader);

This gets us most of the way, but not quite all the way there. This replacement implementation only checks the permission, but we still need to actually modify the accessibility of the object after the permission is granted. This cannot be done directly by the replacement as the accessibility is controlled by a private field on the AccessibleObject. We can, however, accomplish this using the method chaining feature of Byte Buddy, which can be used to chain private methods. The private field on the AccessibleObject class is controlled by a private method named setAccessible0 which we can chain as part of our Byte Buddy transformation using the andThen method.

First create a Byte Buddy type pool that includes the bootstrap classloader and your jar that includes the stubs.

final TypePool bootstrapTypePool = TypePool.Default.of(
new ClassFileLocator.Compound(
new ClassFileLocator.ForJarFile(jarFile),

Next get the AccessibleObject.setAccessible0 method using reflections.

Method setAccessible0Method;
try {
String setAccessible0MethodName = "setAccessible0";
Class[] paramTypes = new Class[2];
paramTypes[0] = AccessibleObject.class;
paramTypes[1] = boolean.class;
setAccessible0Method = AccessibleObject.class
.getDeclaredMethod(setAccessible0MethodName, paramTypes);
} catch (NoSuchMethodException e) {
throw new RuntimeException(e);

With those two pieces we can now build the Byte Buddy transformer to match on the setAccessible method and to delegate to the stub and chain to the setAccessible0 method.

AgentBuilder.Transformer transformer = new AgentBuilder.Transformer() {
public DynamicType.Builder<?> transform(
DynamicType.Builder<?> builder,
TypeDescription typeDescription, ClassLoader classLoader) {
return builder.method(

That will take care of replacing the default implementation of setAccessible with our custom implementation and chain to the setAccessible0 method if permission is granted. If an exception occurs, for instance due to permissions not being granted, the chained transformation will exit early avoiding the call to actually change the accessibility.

The final step is to install Byte Buddy as a Java agent and actually perform the transformation. With a SecurityManager installed and this transformation executed, you can now run untrusted code with permissions to modify the accessibility of its own classes and methods, but not standard Java, your own runtime jar or other isolated code.

Instrumentation instrumentation = ByteBuddyAgent.install();
// Append the jar containing the stub replacement to the bootstrap classpath
AgentBuilder agentBuilder = new AgentBuilder.Default()
.ignore(none()); // disable default ignores so we can transform Java classes

In addition to everything above, you would also want to transform the other setAccessible method, setAccessible(AccessibleObject[] array, boolean flag). For more information on this and a full implementation of a simple sandboxed runtime with this transformation have a look at my sandbox-runtime on Github.

Staying Motivated on Side Projects

I have worked on many different side projects, open source libraries and user facing services/products. While working on these projects, I have found that I sometimes struggle with staying motivated, particularly with the user facing products. I have several completed open source libraries published on GitHub, including Motif, process-warden, dagger-servlet.

My friend Nic and I started a new user facing side project this year, and I took some time for a retrospective look at our past user facing side projects prior to starting. We made some adjustments to how we went about the work that needed to be done, and it has been a success. This is the first side project that I recovered from a motivation slump and continued working on the project with a renewed vigor.

Looking back, it seemed like on past projects the biggest motivation struggle for me was that we started with what I would describe as the “fun” partb followed by the more business-oriented tasks. We would start by focusing on the actual idea, be it a mobile app or a backing service. Once the idea itself was implemented, we would start work on the other necessary tasks like user registration/authentication and marketing that must be done to launch a product. While I do still find these tasks interesting and valuable to learn, I struggle more when I run into a roadblock with these. If the roadblock takes a long time to resolve, I eventually lose motivation, and will take a break from the project. I’ve never recovered from one of these motivation slumps because I know when I begin working on the project I have the same set of tasks to accomplish; there is no proverbial carrot on a stick for me.

This time, we decided to take a different approach. We put the implementation of the idea itself on the back burner and immediately focused on the more business oriented tasks. We began with implementing user registration/authentication. Next, we began working/researching marketing and setting up a pre-launch landing page for beta testers. I hit a motivation slump during September, largely due to the start of Diablo III Season 4. The slump continued a couple weeks beyond when I stopped playing Diablo, but eventually my motivation returned and we kept working toward our goals. Keeping the “fun” work as a carrot on a stick has made a huge difference for me, and I will continue to approach future projects in the same way.

Another advantage of approaching the project this way is that we actually finished the implementation of user registration/authentication, and we can split that out into a reusable component that we can use in future projects. We will no longer have to try to implement that for each project we start.

Our project is called Geolode. It is a service that mobile app developers can use to provide geo-location based user interactions within their apps. This allows app developers to provide very powerful and useful features to their users without having to focus on the development of all of the server side for geo-location based queries. The app developers can focus on what they know best, and we will focus on what we know best.

We have our initial landing page up at If you are interested in becoming a beta tester or even an alpha tester, check it out and sign up for early access. You can also follow project blog at

Implementing Scala-like Pattern Matching in Java 8

I’ve been writing Scala off and on for the last several years. I took both of the Scala courses on Coursera and really enjoyed them. However, I still write a lot of Java. There are a lot of Scala features that I miss while working with Java, and as Java 8 was coming out in early 2014, I thought about feature parity between the two languages. The addition of lambdas to Java is great, but one of my favorite features of Scala is pattern matching, and Java 8 does not include that. At the time I started to wonder if pattern matching could be implemented as a Java library. I wasn’t sure if it would even be possible, but I wrote up a quick Github gist with what some different patterns might look like.

My gist from early 2014 includes Guava Optional matching (I hadn’t yet noticed that Java 8 included it’s own Optional type), Integer matching as an example of matching on exact values, Cons like matching on a list, and the concept of an orElse case. I wrote these examples directly in the gist editor, I’m quite sure they don’t compile, but I wanted to get the concept out of my head and stored somewhere. Once I got the idea out of my head, I promptly forgot about it for just over a year.

In May 2015 I was starting to use Java 8 on some real projects, and one day that old gist popped into my head. I started thinking about it more and decided it might be worth giving a more in-depth look. Initially, I did some searches to see if any libraries had already been created for this. I found a couple that had some pattern matching features, but they were not exactly what I had imagined. Javaslang, provides a lot of nice functional features that are missing from Java 8, but pattern matching isn’t the focus of this library, and it only has support for matching on type and value. Benji Weber gives examples of pattern matching on his blog and implemented them in his expressions library.

The expressions library pattern matching was closer to what I had in mind. It can extract top-level and nested values, but it requires implementing a ‘constructor’ method for anything you want to match on and doesn’t built-in matching for things like Optional or List. At this point I decided I’d give a shot at implementing pattern matching in Java 8 that is as close to the Scala pattern matching as possible. I like to try to come up with a good name for my projects, so I spent some time with a thesaurus looking for words related to ‘matching’ and ‘patterns’, and I eventually landed on motif.

Before I go into some details about the process of implementing motif, below are some examples of what code using motif might look like.

Matching on Optional:

Optional<Person> personOpt = getPerson();
.when(some(any())).then(person -> doStuff(person))
.when(none()).then(() -> System.out.println("Person not found"))

Nested matching:

Optional<Tuple2<String, String>> opt = Optional.of(Tuple2.of("first", "second"));
.when(some(tuple2(eq("third"), any()))).then(b -> doStuff(b))
.when(some(tuple2(any(), eq("second")))).then(a -> doStuff(a))
.when(none()).then(() -> System.out.println("Tuple not found"))

Checkout the Github page or Github repo for additional details. Version 0.1 of motif is available from maven central. It is a little more verbose (horizontally) than Scala, but it provides much of the same functionality. Currently, it supports extracting a maximum of 3 parameters, whereas Scala supports up to 22 in 2.10 and possibly unlimited in 2.11+? I’ve rarely found myself needing to extract more than 3 parameters in Scala, so I thought this would be an OK limitation for now. Support for additional parameters could be added in the future, but it will require writing additional code generation logic.

Motif provides many useful pattern matchers out of the box, but it is also very extensible. A Guava extension could be created, for instance, that provides matchers for Guava’s Optional and any other Guava concepts that would be useful to match on. The motif-generator project provides builders for generating matcher cases for extensions. OptionalCasesGenerator is a good example for using the generator.

The implementation of the Motif pattern matching happened in three phases. The initial phase was a varargs method that accepted the matching cases, and automatically executed the matching. This ran into ambiguity issues for consumers of the code, so I ended up scrapping it and converting to a fluent interface. By using a fluent interface, I could use intermediate classes to track types and avoid ambiguity. This second phase led to what is essentially the final API, except without nested extraction. Once I saw that the fluent interface was working well, I added nested extraction as a third phase. The nested extraction leads to a bit of combinatorial explosion of the API, so I added code generation during this phase to reduce the likelyhood of errors and to avoid writing very similar code multiple times. The code generation portion, is a separate library, and makes it easy for third party extensions to generate all of the necessary methods needed.

Motif has been released as version 0.1 and is available in Maven central. The API is pretty solid, and I don’t expect any dramatic changes in the future. Give it a try in your Java 8 code.

A Safer Way To Create Java Subprocesses

If you have ever had to start a subprocess from a Java application, then you likely know how painful it can be to use the Process class via Runtime.exec or ProcessBuilder. There are numerous posts on stackoverflow as well as a multitude of blog posts on the correct usage of the Process class. I found a blog post on the five common pitfalls of the Process class to be the most helpful when I began using subprocesses. Being aware of the pitfalls helps, but if you are running a lot of subprocesses, the additional code needed in each instance can become tedious and you increase the odds of making a mistake. I ended up creating the process-warden library for a safer way to create Java subprocesses that is reusable.

I make use of subprocesses in the Stackify monitoring agent for executing shell commands as well as for updating a running agent – which I hope to write a separate blog post on at some point. It quickly became clear that having a common way of providing a timeout for the Process.waitFor method and making sure that the streams are closed would be very beneficial. I created the FinalizedProcess and FinalizedProcessBuilder classes that wrap the usage of the Process class and take care of many of the common pitfalls.

FinalizedProcess implements Closable and in its close method it makes sure that all of the subprocess streams are closed as well as optionally destroying the subprocess. It also has a waitFor method that takes a timeout in milliseconds instead of a no parameter waitFor. If the subprocess execution takes longer than the timeout, then an InterruptedException will be thrown and the interrupted flag will be cleared when InterruptedExceptions are thrown. With these additions three of the five common pitfalls are remedied.

For the two remaining pitfalls the problem of process grandchildren not being kill when a subprocess is killed cannot be remedied via code, so that will still require user knowledge to avoid. As for the prompt consumption of subprocess output, I didn’t have an immediate need for discarding the output, so the initial version of process-warden doesn’t provide any functionality for consuming the output. In a future release I hope to add a class that will silently consume the output and error streams along with a method on FinalizedProcessBuilder to enable the silent consumption.

So how do you use process-warden? It’s easy.

Add it to your maven project:


Use FinalizedProcessBuilder to create a FinalizedProcess to execute:

// With Java 7
FinalizedProcessBuilder pb = new FinalizedProcessBuilder("myCommand", "myArg");
try (FinalizedProcess process = pb.start()) {
int returnVal = process.waitFor(5000); // 5 second timeout
// With Java 6
FinalizedProcessBuilder pb = new FinalizedProcessBuilder("myCommand", "myArg");
FinalizedProcess process = pb.start();
try {
int returnVal = process.waitFor(5000); // 5 second timeout
} finally {

If you are currently using Process and ProcessBuilder, give process-warden a shot. It should help simplify your code. I appreciate any feedback or suggestions. Feel free to log issues or fork the project on github.

Differentiate Java Processes by Working Directory

A lot of server software runs on Java, but the different Java processes are listed as simply Java via commands like top or ps -ef. One feature of the Stackify agent is tracking the running processes on a server. I needed a way to identify what each Java process really was in a fairly simple and concise way, but there isn’t a clear winner in the output of the top or ps commands.

Running ps -ef | grep java will output something nasty, similar to what follows. You can likely figure out what the Java process is actually running from the args — the example below obviously being tomcat — however, it’s not the most readable and doesn’t give a clear and concise value to display to a user in the Stackify web site. The user of the process below also indicates that it is running tomcat, but that won’t be the case for all Java processes.

tomcat7 41298 1 5 18:05 ? 00:00:03 /usr/lib/jvm/default-java/bin/java -Djava.util.logging.config.file=/var/lib/tomcat7/conf/ -Djava.awt.headless=true -Xmx128m -XX:+UseConcMarkSweepGC -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.endorsed.dirs=/usr/share/tomcat7/endorsed -classpath /usr/share/tomcat7/bin/bootstrap.jar:/usr/share/tomcat7/bin/tomcat-juli.jar -Dcatalina.base=/var/lib/tomcat7 -Dcatalina.home=/usr/share/tomcat7 org.apache.catalina.startup.Bootstrap start

I decided it might work to differentiate the Java processes by working directory, but I wasn’t sure if there was a good way to find the working directory. Through some Google searches I discovered the pwdx command. The pwdx command takes a list of pids as parameters and outputs a row with the pid and the working directory for each pid. The working directory doesn’t work for all situations, but I’ve found it generally gives a concise way of determining what a Java process is actually running. I’m certainly open to suggestions if anyone has a better idea.

Running the pwdx command for the output above — sudo pwdx 41298 — leads to the following output. By combining this with the process name I can give the Java processes a more informative name when displayed in the Stackify website. This Java process would be displayed as Java (/var/lib/tomcat7).

41298: /var/lib/tomcat7

Drumming as Exercise Routine

I wanted to get into a regular exercise routine, but I have a really hard time getting into a routine doing something I really don’t enjoy. In the past I’ve tried to get into running on my treadmill. Running on the treadmill is really hard for me – my mind starts to wander and I get utterly bored after only a few minutes. Even with music or Netflix to distract me while I run, I just can’t do it. My attempts to get into a running routine typically last around one week. I have decided to run an experiment of drumming as exercise routine, and see what happens. I have a lot of respect for people with the discipline to keep up an exercise routine.

Before I go into more detail about the drumming as exercise routine, first a little history. I got to thinking about how I was able to stay in shape during college. I never made any point to exercise in college and the food I ate was by no means healthy. Most of my time in college, I also drank Coke as my main liquid, drinking almost no water. I probably had a higher metabolism while in college, which contributed to staying in decent shape. I also played a lot of guitar hero and later Rock Band. Guitar Hero certainly isn’t going to burn as many calories as running or cycling, but standing for hours on end, on a daily basis, certainly burns more calories than sitting and watching TV or playing on the computer. The calorie burning was kicked up a notch when Rock Band was released in 2007 and I moved on to playing Rock Band drums for hours on end.

When Harmonix released Rock Band, I already owned an electronic drum set. With the help of the Rock Band forum community, we figured out a way to rig up a midi module with outputs soldered to the Rock Band drums so that we could play Rock Band with a real electronic drum set. With the upgrade to a real drum set, I was playing hours of drums everyday for the last semester and a half of college. I think the full drum set experience made a big difference toward staying in shape.

After college I continued to play drums on a fairly regular basis for a couple of years, but I eventually slowed down and completely stopped. Shortly before Thanksgiving last year I started to think about drumming again for exercise. I started by doing some Google searches in hopes to find some evidence that drumming is decent exercise. I found a lot of typical forum posts with people arguing back and forth, but to my surprise I found an actual scientific study, done by the University of Gloucestershire and the University of Chichester. A lot of the technical details of the study are available on the Clem Burke Drumming Project website, and the BBC also have an interesting article about the study indicating that professional drummers burn between 400 and 600 calories an hour.

Based on the research and my desire to exercise again I decided to start up a regular drumming routine again. Yes, I’m playing Rock Band 3 when I’m exercising, but I’m doing so on a real electronic drum set, and I choose songs mostly from the Nightmare/Impossible difficulty level. I’m definitely not burning 400-600 calories an hour, but I’d guess I’m burning somewhere between 150-300 calories per session. My goal is to play for 45 minutes or more at least 3 days a week. It’s been almost two months since I started and so far I have lived up to this goal. On the weekends I sometimes find myself playing 2 hour sessions.

Generally I create a playlist of 10 songs and start rocking out. By picking the songs ahead of time and creating a playlist it creates the smallest amount of downtime between songs; this way my heart rates stays elevated throughout the play session. I’m not any good at measuring my heart rate, but it is definitely elevated while drumming (though not nearly as much as when I run). I hope to get a digital heart rate monitor, so that I can get a better idea of my heart rate while playing, and at least make some back of the envelope calculations on how many calories I’m burning.

Around the same time I started my drumming exercise routine my girlfriend started a more traditional exercise routine. We have turned it in to a little competition. We are both going to continue with our separate exercise routines and in May we are going to run a 5K. We will determine which exercise routine is better based on who finishes first. Realistically both exercise routines are good, but it’s always fun to add a little competition as additional motivation.

Writing a Good Linux Init Script

A good Linux init script that works across multiple distros is hard to find. Most distros have a sample/skeleton init script to base your own init scripts on, but that requires having a separate init script for each distro. For the Stackify Linux agent I wanted to have a single init script that would work on all common Linux distros. I had a decent init script that worked pretty well for ‘start’ and ‘stop’ commands, but not so much for ‘status’. The ‘status’ command worked on my development machine (Ubuntu) but didn’t work on the other distros I was testing on (CentOS/Redhat/Amazon).

Since one of the things the Stackify Linux agent does is provide information on the different installed services and their statuses, it is important that the status of our own service is correct. As we prepare for the first beta release of the Linux agent, I decided it was time to rewrite the init script. I started with some Google searches on how to create a cross-distro init script, but they mostly led me back to the skeleton init scripts. While having lunch with a former coworker at Fuzzy’s Taco Shop – which I highly recommend – we got to discussing init scripts and he mentioned seeing an “init script to rule them all.”

After lunch, I promptly searched for this legendary script. My search lead me to the One Java init script to rule them all, which is an init script for Java application servers (e.g. Tomcat and JBoss) that will work on most Linux distros. Turns out this script was really close to what I needed since the Stackify Linux agent is a Java application. Many Linux distros have started implementing the Linux Standard Base (LSB) which provides logging methods and a standard way to implement init script statuses. However, since not all versions support LSB, I cannot use those logging methods for my init script.

The nice thing about the Java init script to rule them all is that it checks for the existence of the LSB methods and if they do not exist it provides its own implementation. I was able to base the Stackify agent init script largely on this existing script. The script now works on all of the distros previously mentioned and should also work on most other distros. The status returns the correct status on these distros and returns the correct status code for systems that do support LSB.

My Favorite Music of 2012

I am by no means a music critic, but typically at the end of the year I put together a list of my favorite music from the past year, albeit normally within the privacy of my own head. This year I decided I would make the list public. My musical tastes seem to be pretty eclectic and all over the board. Sometimes the albums that make it onto my list are a surprise even to myself. This year there are 2 albums on my list that I initially did not like, but have grown a lot on me. So without further adieu, here is my favorite music of 2012, in no particular order:

Exitmusic - Passage

Andrew Bird - Break It Yourself

Fun. - Some Nights

Godspeed You! Black Emperor - ‘Allelujah! Don’t Bend! Ascend!

State to State - State to State

Shearwater - Animal Joy

The Lumineers - The Lumineers

Balmorhea - Stranger

Trampled By Turtles - Stars and Satellites

Imagine Dragons - Night Visions

The Exitmusic album is by far my favorite of the year. I only just discovered it mid-December and I have already listened to it more times than any of the other albums on my list. Apparently I really enjoy dark, brooding, depressing albums, as evidenced by this album and my favorite album of 2009, The Antlers - Hospice.

Blogging is Hard

I have decided to give blogging another go. Over the years I have tried to get into blogging on varying topics including music and philosophy. This will be my personal blog, with no specific topic, other than whatever random things enter my head. To keep the blog going, I will make it my goal to create at least one new blog entry a month. Upcoming post topics will likely include Pusher (specifically from a Java consumer standpoint) and my current exercise regimen.