Recursive Resource Gathering in Java

Introduction

Searching the Internet for Class.getResource yields many links discussing issues with Java's resource gathering functions. This article provides code that allows a programmer to locate their resources by scanning the class path, either relative to a root-class or in its entirety, and building a list of resources matching the specified patterns.

The Problem

Class.getResource requires the programmer to know the path and name of any resource they require. In some cases however, requirements may call for resources to be gathered by recursive scanning of the class path.

Programmers quickly discover Java does not have out-of-the-box support for recursively scanning the class path when they learn first hand that Class.getResource and ClassLoader.getResources were not designed with recursive scanning in mind. The problem is made worse considering project deployment with the class path being a simple file system, or a set of JAR files, or a mixture of both!

The Solution

There is a use case where Java allows recursive scanning. It happens when the java.io.FileFilter interface is used to accept or discard files while scanning the file system using the java.io.File class. Let us therefore start with a similar filter interface, but this interface will accept or discard URLs instead of files since resources are often treated as URLs (or InputStreams when they are opened) by the Java platform.

The ResourceURLFilter interface is very simple:

import java.net.URL;

public interface ResourceURLFilter {
  public boolean accept(URL resourceUrl);
}

Next, a class must be designed to utilize ResourceURLFilter instances to collect URLs:

import java.io.*;
import java.net.*;
import java.util.*;
import java.util.jar.*;
import java.security.*;

public class Resources {
}

The Resources class will require access to the Java libraries that handle I/O, URL, and JAR files, but also to a class in the Java Security package; more on this later.

Working backwards, a method to collect a single URL can be added to the Resources class:

import java.io.*;
import java.net.*;
import java.util.*;
import java.util.jar.*;
import java.security.*;

public class Resources {
  private static void collectURL(ResourceURLFilter f, Set<URL> s, URL u) {
    if (f == null || f.accept(u)) {
      s.add(u);
    }
  }
}

The method is declared private and static as it is a helper method and should not be directly used by the programmer. The idea is to consult a ResourceURLFilter instance and if no filter was provided or the filter accepts the URL, add it to the provided set.

The following two methods require a bit more code since they iterate through the folders on the File System and through JAR files when they are a part of the class path. Here is the file system method:

import java.io.*;
import java.net.*;
import java.util.*;
import java.util.jar.*;
import java.security.*;

public class Resources {
  private static void collectURL(ResourceURLFilter f, Set<URL> s, URL u) {
    if (f == null || f.accept(u)) {
      s.add(u);
    }
  }

  private static void iterateFileSystem(File r, ResourceURLFilter f, 
  	Set<URL> s) throws MalformedURLException, IOException {
    File[] files = r.listFiles();
      for (File file: files) {
        if (file.isDirectory()) {
	  iterateFileSystem(file, f, s);
        } else if (file.isFile()) {
	  collectURL(f, s, file.toURI().toURL());
	}
      }
    }
  }
}

Next, we add the method to iterate through JAR file entries:

import java.io.*;
import java.net.*;
import java.util.*;
import java.util.jar.*;
import java.security.*;

public class Resources {
  private static void collectURL(ResourceURLFilter f, Set<URL> s, URL u) {
    if (f == null || f.accept(u)) {
      s.add(u);
    }
  }

  private static void iterateFileSystem(File r, ResourceURLFilter f, Set<URL> s) 
                 throws MalformedURLException, IOException {
    File[] files = r.listFiles();
      for (File file: files) {
        if (file.isDirectory()) {
	  iterateFileSystem(file, f, s);
        } else if (file.isFile()) {
	  collectURL(f, s, file.toURI().toURL());
	}
      }
    }
  }

  private static void iterateJarFile(File file, ResourceURLFilter f, Set<URL> s) 
          throws MalformedURLException, IOException {
    JarFile jFile = new JarFile(file);
    for(Enumeration<JarEntry> je = jFile.entries(); je.hasMoreElements();) {
      JarEntry j = je.nextElement();
      if (!j.isDirectory()) {
        collectURL(f, s, new URL("jar", "", 
        	file.toURI() + "!/" + j.getName()));
      }
    }
  }
}

The two methods use the collectURL(...) method to handle JAR file entries or files on the file system, but another method is needed to direct the flow to either method according to the inspected entry:

import java.io.*;
import java.net.*;
import java.util.*;
import java.util.jar.*;
import java.security.*;

public class Resources {
  private static void collectURL(ResourceURLFilter f, Set<URL> s, URL u) {
    if (f == null || f.accept(u)) {
      s.add(u);
    }
  }

  private static void iterateFileSystem(File r, ResourceURLFilter f, Set<URL> s) 
          throws MalformedURLException, IOException {
    File[] files = r.listFiles();
      for (File file: files) {
        if (file.isDirectory()) {
	  iterateFileSystem(file, f, s);
        } else if (file.isFile()) {
	  collectURL(f, s, file.toURI().toURL());
	}
      }
    }
  }

  private static void iterateJarFile(File file, ResourceURLFilter f, Set<URL> s) 
          throws MalformedURLException, IOException {
    JarFile jFile = new JarFile(file);
    for(Enumeration<JarEntry> je = jFile.entries(); je.hasMoreElements();) {
      JarEntry j = je.nextElement();
      if (!j.isDirectory()) {
        collectURL(f, s, new URL("jar", "", 
        	file.toURI() + "!/" + j.getName()));
      }
    }
  }

  private static void iterateEntry(File p, ResourceURLFilter f, Set<URL> s) 
          throws MalformedURLException, IOException {
    if (p.isDirectory()) {
      iterateFileSystem(p, f, s);
    } else if (p.isFile() && p.getName().toLowerCase().endsWith(".jar")) {
      iterateJarFile(p, f, s);
    }
  }
}

We can now add a set of public methods which will form the API for the Resources class:

import java.io.*;
import java.net.*;
import java.util.*;
import java.util.jar.*;
import java.security.*;

public class Resources {
  private static void collectURL(ResourceURLFilter f, Set<URL> s, URL u) {
    if (f == null || f.accept(u)) {
      s.add(u);
    }
  }

  private static void iterateFileSystem(File r, ResourceURLFilter f, 
          Set<URL> s) throws MalformedURLException, IOException {
    File[] files = r.listFiles();
      for (File file: files) {
        if (file.isDirectory()) {
	  iterateFileSystem(file, f, s);
        } else if (file.isFile()) {
	  collectURL(f, s, file.toURI().toURL());
	}
      }
    }
  }

  private static void iterateJarFile(File file, ResourceURLFilter f, Set<URL> s) 
          throws MalformedURLException, IOException {
    JarFile jFile = new JarFile(file);
    for(Enumeration<JarEntry> je = jFile.entries(); je.hasMoreElements();) {
      JarEntry j = je.nextElement();
      if (!j.isDirectory()) {
        collectURL(f, s, new URL("jar", "", 
        	file.toURI() + "!/" + j.getName()));
      }
    }
  }

  private static void iterateEntry(File p, ResourceURLFilter f, Set<URL> s) 
          throws MalformedURLException, IOException {
    if (p.isDirectory()) {
      iterateFileSystem(p, f, s);
    } else if (p.isFile() && p.getName().toLowerCase().endsWith(".jar")) {
      iterateJarFile(p, f, s);
    }
  }

  public static Set<URL> getResourceURLs() throws IOException, URISyntaxException {
    return getResourceURLs((ResourceURLFilter)null);
  }

  public static Set<URL> getResourceURLs(Class rootClass) 
  	throws IOException, URISyntaxException {
    return getResourceURLs(rootClass, (ResourceURLFilter)null);
  }

  public static Set<URL> getResourceURLs(ResourceURLFilter filter) 
  	throws IOException, URISyntaxException {
    Set<URL> collectedURLs = new HashSet<>();
    URLClassLoader ucl = (URLClassLoader)ClassLoader.getSystemClassLoader();
    for (URL url: ucl.getURLs()) {
      iterateEntry(new File(url.toURI()), filter, collectedURLs);
    }
    return collectedURLs;
  }

  public static Set<URL> getResourceURLs(Class rootClass, 
            ResourceURLFilter filter) throws IOException, URISyntaxException {
    Set<URL> collectedURLs = new HashSet<>();
    CodeSource src = rootClass.getProtectionDomain().getCodeSource();
    iterateEntry(new File(src.getLocation().toURI()), filter, collectedURLs);
    return collectedURLs;
  }
}

Using the Code

The Resources class can now scan the class path and report back with all available URLs, or those matching a specified filter. Please note however that class path scanning should be performed only when absolutely necessary, even if the project is deployed as a set of JAR files.

To scan the entire class path and return all its resources as URLs, simply invoke the getResourseURLs method:

for (URL u: Resources.getResourceURLs()) {
  System.out.println(u);
}

To scan the class path starting with the location from which a specific class was loaded, provide the getResourseURLs method with the root-class:

for (URL u: Resources.getResourceURLs(Resources.class)) {
  System.out.println(u);
}

Things get more interesting when you specify a ResourceURLFilter:

for (URL u: Resources.getResourceURLs(Resources.class, new ResourceURLFilter() {
  public @Override boolean accept(URL u) {
    String s = u.getFile();
    return s.endsWith(".class") && !s.contains("$");
  }
})) {
  System.out.println(u);
}

Things to Consider

• Since the cost of scanning the entire class path — something that can happen even if you do restrict the scan with a root class and a filter — can be expensive, it is advisable to use this code only when required.
• Specifying a root-class to scan from will effectively limit the scan to the JAR file the root-class was loaded from, if and only if, the project actually is deployed as a set of individual JARs. This also means a scan may not find resources if they are not located together with the root-class!
• Although it is impossible to limit the scan to the package of the root-class, it is possible to filter the URLs based on the package of the root-class.
• The code could be extended to support multiple filters, each with its own set of results, but I will currently leave this as an exercise for the reader.
• Currently I am not sure how this code will operate inside Application/Web Servers so I may further update the code if needed.

History

• 19/12/2012 — Article created