Article in Multimedia Systems Design
Return to peter's writing > technical articles

This article was slated for publication in March of 1999, but the magazine folded after the February 1999 issue.

Using Java in Embedded Multimedia Devices

This much is clear: Java holds tremendous promise for multimedia systems. It is the programming language of choice for Web applications, it has gained increasing acceptance over the past several years, and as a programming language, it is designed for and ideally suited to the multimedia environment. That said, it still has some way to go before it fully lives up to its promise. This article discusses the two primary uses of Java in multimedia systems, the architectures that are ideal for those uses, and the pitfalls of implementing those architectures.

Two Uses for Java

Java can be used for applications running inside the device or outside the device, depending on the application's purpose. Typically, Java applications running inside the device will be used much as Java applets are used on the Web today: The device will be displaying Web content and using Java as the programming language for complex tasks and for user interactivity. Java inside the device can also be used for "agent" functionality-small programs that run locally on the device to perform an action on behalf of a remote user or application. Java running outside the device is particularly useful for remote access and control; similar to the agent functionality described above, this type of application is normally used to control and manage the devices remotely.

For example, a low-end Web terminal, possibly in the form of a kiosk or a set-top box, is likely to have a browser built into it. Part of the browser's functionality is the ability to download and run Java applets locally on the terminal. These Java applets or applications can be used to interact with the user, initiate other application functions, or collect and report on local data. For example, a kiosk in a supermarket may use Java applications or applets to collect information from the user, to connect to the supermarket's inventory database or to a local CD-ROM if the network connection is down, and to print out coupons and recipes that are appropriate to the user. Java running in the kiosk allows all that functionality to remain localized within the kiosk (or set-top box or Web terminal) and also allows a level of functionality and control not available through other browser technologies (HTML, JavaScript, and e-mail).

That same kiosk or set-top box also needs to be accessed and controlled remotely. For example, the IT department in charge of the supermarket chain must be able to check the status of each kiosk, upgrade the embedded software, and run remote diagnostics and configuration sequences. In this situation, Java can play two separate roles: If Java is running inside the device, the IT manager can send Java-based agents to run locally on the kiosk, where they can execute commands, interrogate the embedded application, and return with the proper feedback to the IT manager. More appropriate for this type of task, however, and far simpler, is an approach that leverages the Java Virtual Machine (JVM) in the IT manager's browser on his workstation. In this approach, applets are downloaded from the kiosk that present the relevant information and communicate back to the kiosk when necessary. This approach uses a brokered Java architecture as described below. It removes the need to embed a JVM or, if the JVM is part of the browser, to customize that JVM to allow Java applications to control the device-something that could introduce security holes.

Embedding a Java Virtual Machine

As noted earlier, the primary reason to embed a JVM is if it is part of a browser or if the device is expected to run Java applications locally. Because most embedded applications today are coded in C or C++, it's unlikely that we'll see many embedded devices created solely in Java over the next 18 months. Java does, however, offer significant benefits for user interaction and local application functionality:

Running Java in the device, though, incurs several architectural and overhead penalties which may be roadblocks to implementation. First among these is the size of the JVM itself: phrases such as "as low as 1.5 megabytes" are associated with the JVM. Sun Microsystems recently announced the Java Embedded Server (JES), which fits in at just around 600 Kbytes including the basic services of HTTP, SNMP, and threads management.

Figure 1

If the size of the embedded JVM is not a concern, however, there is still the architectural aspect of security and the combination of C code with Java code. A typical device will look something like Figure 1. This figure shows three layers in the device: The communication layer handles all external network connections, communication protocols, etc. The RTOS layer handles basic system services and driver features. The middle layer is the application layer, where both the embedded application and the Java code reside. The problem occurs when the embedded application and the Java applications need to interact; unless the embedded application is written entirely in Java, a special integration layer needs to be written to allow Java applets access to the functions and data embedded in the C application. In the case of a JES, this customized integration layer is referred to as a services bundle.

While this integration layer (represented by the black diamond in the diagram) is not intensely difficult to create, it does add a layer of complexity. Additionally, it can open up security holes in the application if the developer is not careful. If entry points into the embedded application are given to friendly Java applications, precautions have to be taken to ensure that hostile applets and applications are not allowed to take advantage of those entry points. (The JES allows for signatures to be attached to Java ARchive (JAR) files, ensuring that only applets from trusted sources can be run.) Furthermore, as the JVM evolves, maintenance of this integration layer can potentially be complex.

An additional concern about embedded Java is the lack of a true real-time implementation. Because Java runs in a virtual machine, the virtual machine is required to manage its own memory, which it does with a garbage collection thread. This thread can wake up and disturb real-time threads at inopportune moments, something that is deadly to many applications.

Finally, one more concern about Java: Although Java is meant to be run everywhere, it is clear that one size does not fit all. This is manifest in the fact that there are different specifications for Java for different applications. PersonalJava is the specification for Java to be run in embedded devices and is the version implemented in JES. This means that although a multitude of Java applications and applets can be downloaded and run on the device, some of those applications may expect the existence of certain services that may not be available on all devices.

Using a Java Brokering Mechanism

In most cases today, however, a device probably does not need to run Java code locally. While the key benefit to running local Java code is that applets can be downloaded, executed, and discarded, many terminal-style devices will only need Java as part of the browser functionality, including cell phones, PDAs, set-top boxes, etc. Instead, a simpler, smaller architecture that is tailored to embedded development practices can leverage a remote JVM for accessing, controlling, monitoring, and configuring the device itself.

Take the example of a cell phone. Java code can be run inside the device: Applets for initial provisioning and configuration, applets for monitoring, and other applets for dynamic controlling the phone can be downloaded to the phone, run, and discarded as necessary. This incurs the 600 K JES overhead, the overhead of downloading and re-downloading applets, and the complexities of creating customized phone services bundles for the JES. Alternatively, a lightweight "brokering" architecture can be used to expose the internal phone application to external controls.

The key is in the requirements: Remotely provisioning, configuring, monitoring, and controlling the phone, set-top box, or Web terminal do not require Java to be run in the device. Rather, those features require access to and control of the deeply embedded functions and data in the embedded application, and this can be achieved in a much simpler way than embedding a full JVM.

Figure 2

While the embedded JVM is a "heavy server/light client" approach, the brokering architecture is a "light server/heavy client" approach. Applets designed specifically to interact with the embedded application are downloaded from the embedded device as necessary, and they are run in a browser's JVM on the remote terminal. The terminal may be any multimedia type device (including perhaps another cell phone running the JES).

This architecture delivers several key benefits over the embedded JVM approach:

The broker architecture is essentially a client-server mechanism that uses a small integration layer on the embedded device and a Java-based client-side "applet broker" that gets downloaded to the terminal. When the supermarket chain IT manager needs to monitor and update a kiosk, for example, he points his browser at the kiosk, and the kiosk serves up a page with one or more applets in it. These applets are designed specifically for controlling, configuring, and monitoring the kiosk, and they are stored compressed in the kiosk's own file system. In addition, the client-side applet broker is downloaded and run; this broker manages all the communication between the other downloaded applets and the server-side broker. The two sides send information back and forth, allowing complete remote access and control of the kiosk.

Although the brokering approach does not allow any Java code actually to be run within the embedded device, in many cases that is not a requirement anyway. Taking the simpler brokering approach can make development and maintenance of the device easier while providing full Java-based control of the device.


These two approaches for using Java aim at solving different problems. The embedded JVM is intended to download, run, and discard Java applets and applications. The applet broker is intended to allow remote configuration, monitoring, and control of the embedded application. It is critical for the system designer to understand the device's requirements before forging ahead with one or the other. In some cases, an embedded JVM will be required as part of a Web browser; in other cases, embedding a JVM for external management will put too much of a load on the device and is impractical. In either case, using an embedded JVM will require the system designer either to hook the JVM into the embedded application through customized service bundles or to re-build the entire embedded application in Java.

If the ultimate goal is a fully Java-based system, the broker architecture provides a reasonable migration path towards that. Using the broker architecture can get initial products out quickly with critical, required features while the remainder of the system is slowly and methodically ported over to Java.