A program can read its own classes

Fill in a form at a doctor's front desk, and the receptionist can type it into the computer straight away. They've never met you. They don't need to. Every line of the form has its label printed beside it: Name, Date of birth. So they read the label, read what you wrote, and copy it across. Now picture a program that does that with your objects. Objects it has never seen, from classes written after it was. That program exists, and today you write it.

Every big Java framework you'll hear about leans on this one trick. Libraries that turn objects into JSON. Test tools that find your tests without being told their names. Web frameworks that fill in your objects from a web page. People call it magic. It isn't. It's a program reading its own classes while it runs, and the heart of it fits in eight lines.

Fair warning about two things coming later. You'll read a private field from outside its class, and Java will let you. And you'll change a final field, the kind 1.2 promised could never change after its constructor. Java will let you do that too.

What you'll be able to do

• Ask any object for its class, and list that class's fields by name while the program runs.
• Read and write a field you only know by its name — private and final ones included — and say why that's allowed.
• Say what the @ in @Override really is, and write an annotation of your own that changes what a program does.
• Explain how a test tool finds your tests without a list of their names, and build a tiny one yourself.

Section 1The blank form is an object too

In 1.1 you met a class as a blank form, and objects as the filled-in copies. Here's the part 1.1 left out. Java keeps the blank form as well. While your program runs, there's one object in memory for each class it uses. That object describes the class, and you can get hold of it and ask it questions.

That master copy is a Class object. Notice the capital C. Class is a real class in Java, and each of its objects describes one class. Every object carries a ticket for its own Class object, and the method getClass() hands you that ticket. That ticket is what 1.3's hidden note, made by Circle, was all along.

You've met this object already. In 1.4, getClass() showed that a jar of Strings and a jar of Integers share one class. 1.4 also handed methods a token: a class's name followed by .class. Each time it said this chapter would open it up. Here it is, opened.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21// Chapter 1.6, Section 1 — every object can tell you its class.
// Uses: objects and new (1.1), a static nested class (1.2), == on objects (1.1).
public class ClassOf {

    static class Dog {
        String name;
        Dog(String name) { this.name = name; }
    }

    public static void main(String[] args) {
        Dog rex = new Dog("Rex");
        Dog bruno = new Dog("Bruno");
        String word = "hello";

        System.out.println("A: " + word.getClass().getName());
        System.out.println("B: " + rex.getClass().getName());
        System.out.println("C: " + rex.getClass().getSimpleName());
        System.out.println("D: " + (rex.getClass() == bruno.getClass()));
        System.out.println("E: " + (rex.getClass() == Dog.class));
    }
}

The Dog class is 1.1's, unchanged. It's static because it sits inside another class, which 1.2 explained. main builds two dogs and one String.

Line A asks the String for its Class object, then asks that Class object for its name. getName() gives the class's full name. Line B does the same for a dog. Line C uses getSimpleName() instead, the short version 1.4 used.

Line D compares the Class objects of two different dogs with ==. From 1.1, == on objects asks one question: is this the very same object? 1.4 said there's one Class object per class. This line checks it with dogs.

Line E is a new spelling. Dog.class means "the Class object for Dog", written straight from the class's name, with no dog needed. You've typed these before. In 1.4, a class name followed by .class was the token you handed to a method. That token was a Class object all along.

$ java ClassOf
A: java.lang.String
B: ClassOf$Dog
C: Dog
D: true
E: true

$ javac ClassOf.java
$ ls ClassOf*
ClassOf.class           ClassOf.java            ClassOf$Dog.class

A program looking at its own classes while it runs is called reflection. Think of a mirror. The program gets to see itself.

Section 2Print every field of anything, in eight lines

A Class object knows more than a name. It knows every field the class lists. That's the master form's printed lines, and Java hands each one to you as an object of its own.

One strip is a Field object. It belongs to the class, not to any object. It knows a field's name and its type, and nothing about values. The method getDeclaredFields() hands you all of them, in an array.

Here's a class to try it on. It's 0.7's sketch of a Wordle game, made real. All three fields are private, from 1.2, so no other class is meant to see them.

1
2
3
4
5
6public class Game {

    private String secret = "SPEED";
    private int tries = 2;
    private boolean won = false;
}

Game knows nothing about what's coming. It's a plain class. Now a program that has never heard of Game, and takes any object at all:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19import java.lang.reflect.Field;

public class Peek {

    static void peek(Object o) throws IllegalAccessException {
        Class<?> type = o.getClass();
        Field[] fields = type.getDeclaredFields();
        System.out.println(type.getSimpleName() + " has " + fields.length + " fields:");
        for (Field field : fields) {
            System.out.println("  " + field.getType().getSimpleName() + " " + field.getName());
        }
        System.out.println("reading the first one...");
        System.out.println("  " + fields[0].get(o));
    }

    public static void main(String[] args) throws IllegalAccessException {
        peek(new Game());
    }
}

Line by line, because every one is new in some way.

import java.lang.reflect.Field; fetches the Field class. It lives in a package called java.lang.reflect, the way Scanner lived in java.util. Everything reflection uses lives in there.

static void peek(Object o) takes an Object. From 1.2, that's Java's word for "any object at all". peek has no idea what it'll be handed, and that's the point.

throws IllegalAccessException is 1.5's checked-exception rule at work. Reading a field can be refused, and that refusal is a checked exception. So javac makes you say so, here and on main.

Class<?> type = o.getClass(); gets the Class object. The ? is 1.4's wildcard on its own. It means "a Class of some type I don't know in advance". That's the honest truth inside peek.

Field[] fields = type.getDeclaredFields(); asks the class for its strips. An array of Field objects, one per field, from 0.5. The word declared matters. It means "the fields this class writes down itself". Fields it inherits from a parent class, from 1.3, aren't in the list.

The next line prints the class's short name and fields.length, the number of slots in the array.

The for-each loop, from 0.5, visits each Field in turn. For each one it prints two things. field.getType() is the field's type — and it's a Class object again, so getSimpleName() works on it. field.getName() is the field's name, as text.

The last two lines try something bolder. fields[0].get(o) holds the first strip against this one object and reads the answer. Get from this object, the value of this field.

Here's the bet. Those fields are private. Peek is a different class. Private means outside code can't touch them. Or does it?

$ java Peek
Game has 3 fields:
  String secret
  int tries
  boolean won
reading the first one...
Exception in thread "main" java.lang.IllegalAccessException: class Peek cannot access a member of class Game with modifiers "private"
	at java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:392)
	at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:674)
	at java.base/java.lang.reflect.Field.checkAccess(Field.java:1102)
	at java.base/java.lang.reflect.Field.get(Field.java:423)
	at Peek.peek(Peek.java:15)
	at Peek.main(Peek.java:19)

The switch that turns the refusal off

Field has one more method, and it's the one every framework uses. field.setAccessible(true) says: I know this field is private, let me in anyway. Add that, and reading works. That's the whole toy.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19import java.lang.reflect.Field;

public class Dump {

    static void dump(Object o) throws IllegalAccessException {
        Class<?> type = o.getClass();
        System.out.println(type.getSimpleName() + ":");
        for (Field field : type.getDeclaredFields()) {
            field.setAccessible(true);
            System.out.println("  " + field.getName() + " = " + field.get(o));
        }
    }

    public static void main(String[] args) throws IllegalAccessException {
        dump(new Game());
        dump(new ClassOf.Dog("Rex"));
        dump(new Player("Ada", 120));
    }
}

Count the dump method: eight lines, braces included. You've met all but one already. The method gets the Class object and prints its name. Then the loop goes over the declared fields, with no array variable this time. For each one, setAccessible(true) flips the switch. Then the println glues the field's name to whatever field.get(o) reads out of this object.

Then main points dump at three objects. A Game. Section 1's Dog, which ClassOf.Dog reaches inside ClassOf. And a Player, a small class in src/ with a private name, a private score, and a static counter that every constructor adds one to. Dump.java never mentions any of their fields.

$ java Dump
Game:
  secret = SPEED
  tries = 2
  won = false
Dog:
  name = Rex
Player:
  created = 1
  name = Ada
  score = 120

Eight lines, and they read three classes they were never told about. Your Wordle secret, read by a program that has never heard of Wordle.

Look at Player's first line, though. created = 1. That field is static, so from 1.2 it belongs to the class, not to this player. getDeclaredFields() lists static fields too, and dump printed it like any other. Teaching dump to skip it is a job for Your turn.

One more honest point. The fields came out in the order each file wrote them. That's what this JVM does. But the documentation of getDeclaredFields() says the fields are "not in any particular order". So the order is a habit, not a promise. Never write code that needs it.

Here's what happened in memory, one line of dump at a time.

One object it can't read

Everything is an object, so let's be greedy. A String is an object. Its letters must be sitting in a field somewhere. src/DumpString.java calls the very same dump twice: once on a Game, as a control, and once on "hello".

$ java DumpString
Game:
  secret = SPEED
  tries = 2
  won = false
String:
Exception in thread "main" java.lang.reflect.InaccessibleObjectException: Unable to make field private final byte[] java.lang.String.value accessible: module java.base does not "opens java.lang" to unnamed module @59bd9f3b
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
	at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
	at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
	at Dump.dump(Dump.java:11)
	at DumpString.main(DumpString.java:7)

$ java --add-opens java.base/java.lang=ALL-UNNAMED DumpString

Section 3The @ you've been copying since 1.2

Since 1.2 you've typed @Override above every equals and hashCode. 1.2 called it a one-word guard. 1.3 used it on every area, and promised this chapter would explain the @. Here's the explanation.

That tag is an annotation. It's a label you attach to a piece of code: a class, a field, a method. It doesn't change what the code does. A program that goes looking for it can read it, and act on it.

@Override is a tag for javac. It says "this replaces a starter method, or keeps an interface's promise". javac reads it while compiling, and refuses if there's nothing to replace. That's the whole job. Today you make a tag of your own, for your own program to read at run time.

Here's the dump again, taught to keep a secret. Fields marked with a tag called @Secret print as stars.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.FIELD)
@interface Secret {
}

static class Login {
    String user = "ada";
    @Secret
    String password = "letmein";
}

Lines 1 to 4 fetch four names that tags about tags need, from a package called java.lang.annotation. That's 1.2's import line, which fetched HashSet the way 0.3 fetched Scanner. Leave one out and javac says cannot find symbol on the line that uses it.

Lines 8 and 9 declare a new kind of tag, called Secret. The spelling borrows interface from 1.3 with an @ in front. Read @interface as one word that means "a new kind of tag". This one has nothing inside its braces, because it carries no information beyond being there.

Line 7, @Target(ElementType.FIELD), is a tag on the tag. It says @Secret may only be tied to fields. Tie it to a method and javac refuses with annotation type not applicable to this kind of declaration. I tried it.

Line 6, @Retention(RetentionPolicy.RUNTIME), is the one that matters most, and it's about time. It says: keep this tag inside the compiled file, so it's still there while the program runs. Remember the word retention. This section's surprise hangs on it.

ElementType.FIELD and RetentionPolicy.RUNTIME are each one of a short, fixed list of named choices that Java offers. You use them the way you'd use a static final constant from 1.2, like Math.PI. Java's word for a fixed list like that is an enum. This course never needs to write one, so the name is all you need.

Lines 11 to 15 are a small class with two fields. The tag sits on the line above password, exactly where @Override sits above a method.

Now the dump that reads it. It's Section 2's dump with three new lines in the loop.

1
2
3
4
5
6
7
8for (Field field : type.getDeclaredFields()) {
    field.setAccessible(true);
    Object value = field.get(o);
    if (field.getAnnotation(Secret.class) != null) {
        value = "***";
    }
    System.out.println("  " + field.getName() + " = " + value);
}

Line 3 reads the value into a box first, so it can be swapped. Line 4 is the new idea. field.getAnnotation(Secret.class) asks this Field: do you have a Secret tag on you? The Secret.class is a class token, exactly like 1.4's. You get the tag back, or null if there isn't one. Line 5 swaps the real value for stars when the tag is there.

$ java SecretDump
Login:
  user = ada
  password = ***

The class said which field was secret. The dump read the tag and did the right thing. Neither one knows the other exists. That's the deal annotations make, and it's how every framework you'll use asks you for instructions.

Now break it. Open SecretDump.java and delete the whole @Retention(RetentionPolicy.RUNTIME) line. Leave everything else alone. Compile and run.

$ javac SecretDump.java
$ java SecretDump
Login:
  user = ada
  password = letmein

$ java OverrideGone
tags left on toString: 0
tags left on bark:     1

Section 4Building an object from its class

Reading is half the job. Plenty of tools must go the other way. Read some text, then build an object and fill its fields, from outside.

You know how to build an object: new Game(). But look at what that needs. The word Game, typed into your source. A tool like that is written before your classes exist. It can't type their names. It only gets handed a Class object, like Game.class, and has to make one from that.

So it asks the Class object for a constructor and runs it.

1
2
3
4
5
6
7
8
9
10static <T> T blank(Class<T> type) throws ReflectiveOperationException {
    return type.getDeclaredConstructor().newInstance();
}

public static void main(String[] args) throws ReflectiveOperationException {
    Game game = blank(Game.class);
    Field secret = Game.class.getDeclaredField("secret");
    secret.setAccessible(true);
    secret.set(game, "CRANE");
    Dump.dump(game);

Line 1 is a generic method, from 1.4. You hand it a Class<T> and you get back a T. Hand it Game.class and you get a Game, with no cast. That's exactly what the token was for.

throws ReflectiveOperationException names a whole family. Every checked exception reflection can throw is a child of it — IllegalAccessException from Section 2 among them. From 1.5, naming the parent covers all the children.

Line 2 does the work in two calls. type.getDeclaredConstructor(), with empty brackets, asks for the constructor that takes no parameters. That's a Constructor object, the third kind of reflection object. Each kind stands for one part of a class. A Class object stands for the whole class: the master form. A Field object stands for one field: one printed line. A Constructor object stands for one constructor: the instructions for filling in a fresh copy. Then .newInstance() runs it, exactly as new would, and hands back the new object.

But Game.java has no constructor at all. Here's the rule that makes that fine. When a class writes no constructor, javac writes one for you, with no parameters. It sets the fields to their starting values, like "SPEED", and does nothing else. That's the one blank found.

Lines 6 to 10 use it. Build a blank game. Ask the class for one field by name, with getDeclaredField("secret"). Flip its switch. Then secret.set(game, "CRANE") is the write: into this object, put this value. Then dump the game, to see.

$ java Build
Game:
  secret = CRANE
  tries = 2
  won = false

A Game made without the word new anywhere near it, with a private field rewritten from outside. That's everything a tool needs to fill objects from text. It's also a little alarming, so let's push.

A final field

1.2 made a promise about final. A final field gets filled once, by the constructor, and never again — "not even code inside the class". Here's a class that relies on that.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19static class Badge {
    private final String owner;

    Badge() {
        owner = "nobody";
    }

    String owner() {
        return owner;
    }
}

// ...later, in main:
Badge badge = blank(Badge.class);
System.out.println("badge before: " + badge.owner());
Field owner = Badge.class.getDeclaredField("owner");
owner.setAccessible(true);
owner.set(badge, "Grace");
System.out.println("badge after:  " + badge.owner());

Badge is 1.2's pattern exactly. One private final field, filled once by a constructor with no parameters, and a getter to read it. No way to write it.

Then main builds a badge with blank and prints its owner. Then it tries the three lines that rewrote the game's secret.

$ java Build
badge before: nobody
badge after:  Grace

$ java Hidden
WithStatic:    Hidden$WithStatic()
WithoutStatic: Hidden$WithoutStatic(Hidden)

Section 5How frameworks do their magic

Here's the claim from the top of the page, cashed in. Every framework that seems to know about your classes does some mix of four things. You've already done three and a half of them yourself.

1. Find the classes, fields or methods it cares about, with getDeclaredFields() and friends.
2. Read the tags you left, with getAnnotation.
3. Build your objects, with the constructor that takes no parameters.
4. Fill them in or call them, with set, or with a method call made by reflection.

That last one is the only piece you haven't seen. A Class object hands out Method objects as well, one per method, and a Method can be run. Here's the central trick of a testing framework, in one file.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.reflect.Method;

@Retention(RetentionPolicy.RUNTIME)
@interface Test {
}

static class MathTests {
    @Test
    static boolean twoPlusTwo() {
        return 2 + 2 == 4;
    }

    @Test
    static boolean tenthsAddUp() {
        return 0.1 + 0.2 == 0.3;
    }

    static boolean notATest() {
        return false;
    }
}

// ...in main:
for (Method method : MathTests.class.getDeclaredMethods()) {
    if (method.getAnnotation(Test.class) == null) {
        continue;
    }
    run++;
    boolean ok = (Boolean) method.invoke(null);
    if (ok == false) {
        failed++;
        System.out.println("FAIL: " + method.getName());
    }
}
System.out.println(run + " tests, " + failed + " failed");

Lines 1 to 3 are the imports: two from SecretDump, plus Method from java.lang.reflect. Lines 5 to 7 declare the tag, @Test, kept at RUNTIME like @Secret. There's no @Target line this time. Without one, a tag may go on almost any piece of code, methods included.

MathTests has three methods. Two carry the tag, and one doesn't. Each test hands back true if it passed.

The loop is the dump again, with methods. getDeclaredMethods() gives one Method object per method. Any without a @Test tag are skipped, so notATest never runs. run counts the rest.

method.invoke(null) runs the method. The null is where the object would go, and these methods are static, so there isn't one. invoke always hands back an Object, so the (Boolean) cast says what it really is. Then a failure is counted and named.

$ java TinyTest
FAIL: tenthsAddUp
2 tests, 1 failed

Two tests found, without a list of their names anywhere. One failed, and it's 0.2's old friend: 0.1 + 0.2 isn't 0.3 in a double. notATest was never run, so false never counted.

The json-java tests you'll run in 1.7 and 1.8 use a hand-made check method. Most real Java projects use a testing tool called JUnit instead. You mark each test with @Test, and JUnit finds and runs them by reflection. It's the same four steps, done far more carefully.

Once you know the four steps, you'll see them everywhere. Gson, a popular library that turns objects into JSON, has a tag called @SerializedName. Put it on a field, and that field gets a different name in the JSON. Spring, a big framework for building web services, builds your objects for you and fills in fields you mark with @Autowired. None of it is magic. All of it is a program reading its own classes.

Section 6Check yourself

Section 7What broke, and what surprised you

Write this down in your own words before you move on. The predictions you got wrong are the ones worth having. Each one points at a single belief you can now replace.

Worth answering properly, on paper:

1. Which lines did you get wrong, and what were you picturing instead? Name the picture.
2. In one sentence: what's the difference between a Field object and the value in a field?
3. Why could Build write a final field, when 1.2 said nothing could? And name one place reflection is still refused.
4. What did deleting @Retention break, and why did nothing complain?
5. How did TinyTest find its tests without a list of their names?

If a run didn't match this page, trust your terminal and go and find out why. Every output here came from a real run on JDK 17. But your machine is the one that counts.