Project JXTA is a community-run attempt to build a utility application substrate for peer-to-peer applications. The initial reference implementation of JXTA includes a command-line shell that allows experimentation with the core JXTA platform without programming. In this second installment in a three-part series, Sing Li takes us through a hands-on tour of the JXTA shell. You'll explore its command set and extend its capability by writing your own custom commands using the Java programming language.

The JXTA command shell is the very first JXTA application that most JXTA developers will encounter. In fact, the JXTA development kit includes the shell as the default application. This is done for a very good reason: you can gain an appreciation of the components that make up the JXTA platform via the shell -- without writing a single line of code. The shell is a command-line tool that is similar to the familiar UNIX or DOS shell and makes extensive use of environment variables. In this installment of our series, you will become familiar with the essential commands of this versatile shell and learn how to enhance its capability by writing your own commands.

Before we start, you'll need to first install the JXTA shell. You can download the JXTA shell distribution (choose a stable build) from the Resources section at the end of the article.

Setting up JXTA

Installing the JXTA shell Installing JXTA on your machine, at least as of build 29i (the current version at the time of writing), is far from easy or trivial on most home or office systems. The key difficulty comes in the way JXTA works with firewall proxies and the de facto standard home network NAT routers. To interact with other peer nodes out over the Internet, you will need to work out the configuration details following the JXTA installer documentation. For the purpose of this article, however, we can experiment with JXTA without the need to connect outside of our local network. Simply install the distribution and follow the instructions in the main text. In the next and final article of the series, we will discover how to configure our JXTA system for the Internet.

To set up the environment required for the experiments in this article, follow these steps:

1. Download the source code distribution for this article and unzip it in a directory of your choice. Throughout this article, we will call this directory CODEROOT.
   
   
2. Unarchive the JXTA shell binary distribution (jxtashell.zip) into the same CODEROOT directory.
   
   
3. Check the lib directory under CODEROOT and make sure you have jxta.jar and jxtashell.jar in it.
   
   
4. Go into the shell directory under CODEROOT and run the runshell.bat file (included with the code distribution of this article). This will start the GUI config utility, which is shown in Figure 1.
   
   
5. When the GUI config utility requests a peer name, enter node1. Make sure that the TCP transport section is checked, and the hostname or IP address that you enter there is the valid local address of your machine. Also make sure that you have specified TCP port number 9700 for this node. All this is shown in Figure 1 below.
   
   
6. Disable the HTTP transport by unchecking the appropriate checkbox. HTTP transport is used to communicate with peers beyond a firewall (through a rendezvous service); we will not need it.
   
   
7. Click the OK button to start an instance of the JXTA shell.

Back to top

Anatomy of a peer

Time for some experimentation. As is the case in the UNIX shell, commands in the JXTA shell interact with environmental variables. To find out some more information about a peer, try the following command:

JXTA>whoami

Your output will be a structured document in an XML-like syntax:

<Peer>node1</Peer>
<Keywords>Netpeer group by default</Keywords>
<PeerId>
jxta://59616261646162614A78746150325033DB1EB6636DCE4B2990CA888B36CD96C7000
  0000000000000000000000000000000000000000000000000000000000301
</PeerId>
<TransportAddress>tcp://169.254.101.152:9700/</TransportAddress>

Here, we can see the local peer name node1 corresponding to a long peer ID (in the <PeerId> element). Note that we are already in a default peer group, called Netpeer. The transport address used to reach this peer is in the <TransportAddress> element. If we do not specifically modify the shell configuration, the shell booting mechanism will put you into the default peer group. In fact, we can find out some more about this default group with the following command:

JXTA>whoami -g

This variant of the whoami command, using the -g switch, will examine the group of which we are currently a member and obtain information about it. Here is what our output may look like:

<peer group>Netpeer group</peer group>
<Keywords>Netpeer group by default</Keywords>
<peer groupId>
jxta://59616261646162614A757874614D504700000000000000000000000000000000000
  0000000000000000000000000000000000000000000000000000000000201
</peer groupId>
<Service>jxta.service.discovery</Service>
<Service>jxta.service.pipe</Service>
<Service>jxta.service.resolver</Service>
<Service>jxta.service.rendezvous</Service>
<Service>jxta.service.peerinfo</Service>
<Service>jxta.service.membership</Service>

This is the default Netpeer group. Notice the list of all the core peer group services that are available; each service is enclosed in <Service> tags.

Back to top

Basic P2P communications

With the peer up and running and a member of the Netpeer group, we can now try some P2P JXTA communications. We will:

1. Create a pipe
2. Create a message
3. Send the message through the pipe to another peer

Using the commands available on the JXTA shell, we need to break down the above tasks into the following steps:

1. Create a pipe advertisement.
2. Create an input pipe based on the advertisement.
3. Start a new shell instance.
4. On the new shell, create an output pipe based on the advertisement.
5. On the new shell, create a message.
6. On the new shell, attach a file to the message.
7. Put the old shell in blocking mode waiting for message from the input pipe.
8. On the new shell, send the message.

Now, let's go through this procedure step by step and examine all the shell commands that we will use. We'll become acquainted with almost all of the currently available shell commands in the process.

Creating a pipe advertisement

We can create a pipe advertisement and assign it to an environment variable called dwAdv using the following shell command:

JXTA>dwAdv = mkadv -p

The mkadv command is used to create an advertisement, either for peer groups or for pipes. (In this case, the -p switch indicates that this is a pipe advertisement.) At the current stage of JXTA development, the -t switch, which we would use to specify a type of pipe, is not yet functional. Eventually, you will be able to use this command to create different types of pipe advertisements (for example, advertisements for propagate pipes, point-to-point pipes, and so on; see the first installment of this series for more details).

We can see the effect of mkadv by using the following command:

JXTA>env

env lists all the environment variables. You will see that the dwAdv environment variable is now set:

dwAdv = PipeService Advertisement (class net.jxta.impl.protocol.PipeAdv)

Creating an input pipe based on the advertisement

We'll now create an input pipe associated with an environment variable called dwinpipe:

JXTA>dwinpipe = mkpipe -i dwAdv

The -i switch indicates an input pipe, and dwAdv is the pipe advertisement we created earlier.

Use the env command to see that dwinpipe is now associated with the input pipe:

dwinpipe = InputPipe of dwAdv (class net.jxta.impl.pipe.InputPipeImpl)

Starting a new shell

You can spawn a new shell instance, carrying the current environment variables, using the following command:

JXTA>Shell -s

The -s switch creates a new shell, inheriting all of the old shell's environment variables, in a new window. If we did not use the -s switch, a new shell would still be created inheriting all the environment variables, but it would be "nested" or "stacked" on top of the existing shell within the same window. No new window would be displayed, and you would not be able to access the old shell until you exited the new one.

For our example, keep in mind that in both shell instances we will be dealing with the same peer. We'll work with more than one peer in the next section of this article.

Creating an output pipe

In the newly created shell instance, we can create an output pipe associated with the dwAdv advertisement. The command to use is:

JXTA>dwoutpipe = mkpipe -o dwAdv

This new output pipe, accessible through the environment variable dwoutpipe, will be associated with the same input pipe we created earlier, as both are based on the same advertisement (dwAdv). If this pipe were linking two different peers, we would need to advertise the pipe, a topic we'll discuss later.

Creating a message

Now we need to create a message that we can send through the pipe. To do so, we'll first create an empty message associated with the dwMsg environment variable:

JXTA>dwMsg = mkmsg

Attaching a file to the message

Import the text file called simple.txt (included in the shell directory, which is contained in source code supplied in Resources) and associate it with an environment variable called dwData using the following command:

JXTA>importfile -f  simple.txt dwData

The -f switch is mandatory. It indicates that there is a file name immediately following; the usage is similar to the UNIX tar command. Omitting it will cause undefined behavior, including an exception!

Now, attach the imported file as a tagged value (associated with the tag named dwTag) onto the message. Conceptually, we are attaching attributes to the message object. The attributes will ride with the message through the pipe and can be detached at the other end. Underneath, an additional XML snippet, which includes the attached document and is named by the tag value, is inserted into the structured document (message); the associated header is adjusted to reflect the new larger size. This is all achieved using the put command. Later on, the same tagged value can be extracted from the message by means of the get command, which operationally reverses the above process to retrieve the imported file.

To attach the tagged value as an attribute, we use the put command:

JXTA>put dwMsg dwTag dwData

This command will add the text file to the message body associated with dwTag.

Getting ready to receive a message

On the old shell instance, start the blocking receive on the input pipe using the following command:

JXTA>dwNewMsg = recv dwinpipe

This command will set up the shell to wait for a message and associate the dwNewMsg environment variable with any incoming message that is received through the pipe.

Sending the message through the pipe

Back on the new shell instance, we can now send the message with the attached file through the pipe using this command:

JXTA>send dwoutpipe dwMsg

Verifying the message and file receipt

On the old shell instance, where the receiver is blocking, we should see:

JXTA>dwNewMsg = recv dwinpipe
recv has received a message

Now, try env and notice that the dwNewMsg environment variable is now associated with the received message.

dwNewMsg = 
  Message from  dwinpipe (class net.jxta.impl.endpoint.MessageImpl)

In fact, we can easily extract the file from the pipe using the dwTag tag. Use the get command to associate the file with the dwNewFile environment variable:

JXTA>dwNewFile = get dwNewMsg dwTag

To see the content of the transferred file, we can use the following command:

JXTA>cat dwNewFile

The content of the file will be displayed as an XML document:

 <?xml version="1.0"?>
<ShellDoc>
     <Item>
          This is a simple file for JXTA transfer. 
           It can contain anything.
     </Item>
</ShellDoc>

To save the message to a file called received.txt, use this command:

JXTA>exportfile -f received.txt dwNewFile

This will save the file in the directory where the shell was started initially. Again, the -f switch specifies that a filename is following.

Piping between two independent peers

Now that we have actually transferred a file through a pipe, let us recap some of the unique characteristics of the JXTA system:

• A pipe is uniquely identified by its advertisement, which is an identifier embedded in an XML document.
  
  
• When a pipe advertisement is created, no physical pipe or connection is made; the pipe exists but is not bound to any endpoint.
  
  
• You can bind input and output pipes to endpoints independently at any time after creation of the advertisement.
  
  
• The actual transfer of data between the endpoints is typically coordinated by an underlying pipe service within the peer group.

Our example above used only a single peer, or endpoint. We can easily modify it to use two peers within the same default peer group. To try this out, you can either set up another peer instance of the JXTA shell on the same machine as described below, or you can start up the JXTA shell on two independent machines within the same local area network.

The GUI config utility The GUI config utility will appear only the very first time you start a shell instance. Thereafter, the config information is stored on disk and may be reused. This is why we create a shell and a shell2 directory -- to contain two different sets of config data. If you want to go through the configuration process again, you can enter the peerconfig command while you are in the JXTA shell to start the GUI config utility the next time you start the shell instance.

Setting up two independent peers on the same machine
To simulate a P2P network with two nodes, we will set up a second independent peer on the same machine where we set up the first. With this setup, we'll be able to play without having to move between multiple machines on the network. The key to getting this working is to set each instance of the platform to run off a different TCP port. You can do this by following these steps:

1. Change into the shell2 directory under the CODEROOT directory.
   
   
2. Start up the shell by running the runshell.bat file.
   
   
3. At this point, the JXTA GUI config utility will start. Change the TCP port to 9701 (recall that the other shell instance, peer node1, is at port 9700), and enter a new peer name (for this example, use node2).
   
   
4. Disable the HTTP transport (unchecking the appropriate box) and complete the configuration.

The above procedure will start an independent peer running on the same machine where node1 runs but at a different TCP port. Now we can create a pipe and send a message between the two independent peers. Essentially, we will do everything that we did on the earlier single peer case again -- but this time, we'll do it on the new peer instance instead of on a second shell instance.

For convenience, we will call the peer where you create the input pipe by its peer name -- node1 -- and the new peer instance node2. On node1, after you've created the input pipe, you must make the advertisement visible to node2. Environment variables aren't automatically shared across independent peers; this is true for peers on different machines connected over a network, as well as for independent shell instances on the same machine. To publish an advertisement within a peergroup, use the share command:

JXTA>share dwAdv

To see the advertisement on node2, you must perform a remote search:

JXTA>search -r

The -r switch indicates that the command will search advertisements published by remote peers within the same peergroup. Remember, a "remote" peer could be running in an independent shell instance on the same machine.

Next, repeatedly enter the command:

JXTA>search

This action will poll for any newly discovered advertisements. Eventually you will see something like this on node2:

JXTA>search
JXTA Advertisement adv0 

The shell will assign an environment variable named advn to each advertisement discovered, with n starting at 0. In our example, the shell assigns the variable adv0 to the newly discovered advertisement.

Now you can use this adv0 shared advertisement to make the output pipe:

JXTA>dwoutpipe = mkpipe -o adv0

That covers the necessary steps for setting up two peers; all the remaining steps are identical to the single-peer case. Using pipes, it is possible to exchange files, data, and code easily among peers in a peer group.

Back to top

Extending the shell: A simple example

The JXTA shell is designed to be easily extensible by users. This means that you can add custom commands to the shell. Because the reference implementation of JXTA shell is implemented in the Java language, dynamic class loading means that JXTA shell extensions can be added dynamically without recompiling the shell. This, in turn, means that different users can use the same base shell binary but have their own sets of favorite extensions. In fact, you don't even need to have access to the source code of the JXTA shell to write an extension. The best way to illustrate this amazing extensibility is through a trivial example. We are going to add a command to the shell called dwcmd (short for developerWorks command). dwcmd simply prints a text string when executed:

JXTA>dwcmd
This is a trivial JXTA shell extension.

Here is the simple source code of the dwcmd extension:

package net.jxta.impl.shell.bin.dwcmd;
import net.jxta.impl.shell.*;
public class dwcmd extends ShellApp {
  public dwcmd() {
  }

  public int startApp (String[] args) {
     println ("This is a trivial JXTA shell extension.");
     return ShellApp.appNoError; 
  }

  public void stopApp () {
  }

}

We can see that:

• The extension extends the net.jxta.impl.shell.ShellApp class.
• It must implement the StartApp() and stopApp() methods.
• It is part of the net.jxta.impl.shell.bin package.

All shell extensions follow this pattern. The StartApp() method is called when the command is invoked from the shell command line, and the StopApp() is called when the shell exits.

Compiling and configuring shell extensions

To successfully compile the dwcmd.java file, we must add jxtashell.jar in the classpath when the compiler is invoked. This step will allow the Java compiler to resolve both the JXTA core classes and the JXTA shell-specific library classes used in the shell extension code. A makeit.bat file has been supplied in the code directory; it contains:

set CODEROOT=..
javac -classpath 
%CODEROOT%\lib\jxta.jar;%CODEROOT%\lib\jxtashell.jar 
  -d %CODEROOT%\dwclasses net\jxta\impl\shell\bin\dwcmd\*.java

Notice that the dwcmd command is part of a package called net.jxta.impl.shell.bin. In fact, all shell commands are configured in this way. This is how the JXTA shell will find the command (using introspection). It is also the exact mechanism by which extensions are added dynamically to the shell, during run time, without requiring recompilation of the shell itself.

Our makeit.bat compilation process places the output classfiles in a directory called dwclasses. This is a directory that we need to add to the classpath of the shell instance that we will be running. This effectively merges the shell extension classes with the JXTA shell implementation classes during run time. If you look at the runshell.bat file from the shell directory, you will see that the classpath includes it:

java -classpath 
..\lib\jxta.jar;..\lib\jxtashell.jar;..\lib\log4j.jar;..\dwclasses 
  net.jxta.impl.peergroup.Boot

Here, we have added our ..\dwclasses directory to the classpath of the VM running the shell. This effectively merges our net.jxta.impl.shell.bin.dwcmd package with the rest of the shell commands that reside within the jxtashell.jar file. Figure 2 illustrates how this merging of commands works.

Now if you start a new shell instance (using runshell.bat) in the shell directory, you will be able to access the new dwcmd command within the shell. Try this for yourself!

Back to top

Coding a complex shell extension

Now that we are familiar with the basics of creating a shell extension, we can write a more complex extension that actually performs useful JXTA work. In fact, we will create a shell extension that will:

1. Create a pipe advertisement
2. Create an input pipe based on that advertisement
3. Share the advertisement within the group
4. Wait on the input pipe for a message
5. Extract an attached text file from the message
6. Print out the contents of that text file

This is the same sequence that we worked through earlier on node1 using shell commands and shell variables. Now, with the help of the extension, we can do everything within one custom single command.

The new command will be called waitptext (short for wait pipe text). You can find the source code under net.jxta.impl.shell.bin, the subtree where all shell commands must go. Take a look at the import list of waitptext.java below to see the various JXTA platform packages we need to import:

package net.jxta.impl.shell.bin.waitptext;
...
import net.jxta.pipe.*;
import net.jxta.endpoint.*;
import net.jxta.discovery.*;
import net.jxta.document.*;
import net.jxta.protocol.*;

Here is a quick summary of the packages included and the function that each serves:

In the waitptext class, we define private instances of the JXTA object types (advertisements, pipes, messages, structured documents, and the like) that we will work with. On the command line, everything was assigned to shell variables; in JXTA programs, we need to be more type specific.

public class waitptext extends ShellApp {

  private PipeAdvertisement myPipeAdv = null;
  private InputPipe myInpPipe = null;
  private Message myMesg = null;
  private InputStream myInpStream = null;
  private StructuredDocument myStructDoc = null;
  private Discovery myDiscovery = null;
  private PeerGroup myGroup;

The most important method of waitptext.java is startApp(), where most of the work of the command is done. In this case, we make use of the argument being passed into the command. This is the name of the structured document tag (dwTag in our earlier example) that we will look for in order to detach the text document being sent with the message. We determine if the user specified an argument for the message tag name; if not, we use the default (set to tmpTag earlier):

private String tagName = "tmpTag";
public int startApp (String[] args) {
  if (args.length != 0)
    tagName = args[0];

Creating a pipe advertisement

We create the pipe advertisement using the static AdvertisementFactory class, part of the net.jxta.document package (because an advertisement is a structured document). As you'll recall from Part 1 of this series, each advertisement has a unique ID; here, we create it by creating a new PipeID instance. We pass in our current group ID as an argument for construction of the PipeID, because every PipeID contains the ID of the peergroup in which it was created. The UUID-generation algorithm is part of the internal platform core library, and the new PipeID() implementation will call it internally. The PipeID actually contains an embedded group ID, because a pipe always belong to a peergroup. It is actually a composite of two UUIDs -- that's why the group ID is passed in during PipeID construction. Note that group is a protected variable available to every shell application, containing the current peer group for the shell.

  try {
         myPipeAdv = (PipeAdvertisement) 
                     AdvertisementFactory.newAdvertisement(
                     PipeAdvertisement.getAdvertisementType());
         myPipeAdv.setPipeID( new PipeID(group.getID()) );
       } catch (Exception e) {
         e.printStackTrace();
         return ShellApp.appMiscError;
       }
      if (myPipeAdv == null) {
         println("waitptext: Cannot create a Pipe Advertisement");
         return ShellApp.appMiscError;
     }
     println("created a pipe advertisement...");

Creating an input pipe

After creating the pipe advertisement, we must create an input pipe based on the advertisement. This is done using the net.jxta.pipe.Pipe interface. Here, we use the protected pipes variable inherited from ShellApp; pipes always contains the default pipe service for the default group of the shell.

       try {
           myInpPipe = pipes.createInputPipe (myPipeAdv);
          } catch (IOException e) {
           println("waitptext: Cannot create a Pipe");
           return ShellApp.appMiscError;
       }
       println("created an input pipe based on the advertisement...");

Publishing the pipe advertisement

Now we need to make the pipe advertisement visible to other peers in the group. We will use the primitive JXTA discovery protocol to publish the advertisement. In a production P2P application, of course, we can use other means (that is, other services) to circulate the advertisement.

     try {
             myDiscovery = group.getDiscovery();
             myDiscovery.publish(myPipeAdv, Discovery.ADV);
     } catch( Exception e ) {
             println("waitptext: publish of pipe advertisement failed");
             return ShellApp.appMiscError;
     }
       println("published the pipe advertisement to the group...");

Waiting for an input message

After making the advertisement available, we block on the input pipe, waiting for an incoming message. This is done using the poll() method of the Pipe instance. A timeout of 0 indicates that we will wait for the incoming message indefinitely.

           println("waiting at the input pipe for a message...");

    try {

         myMesg = myInpPipe.poll(0);
    } catch (IOException e) {
     return ShellApp.appNoError;
    }

Extracting and printing the attached file

If we reach this point of the code, a message has been received. We then create a structured document based on the data tag specified by the user, and we print the text content of this structured document to the command window.

    try {
        myInpStream = myMesg.pop (tagName);
println("tagName used is " + tagName);
        myStructDoc =  StructuredDocumentFactory.newStructuredDocument (
               new MimeMediaType ("text/xml"),
               myInpStream);
        OutputStream out = new ByteArrayOutputStream();

        myStructDoc.sendToStream( out );
        println("received a message...");
        print ( out.toString() );
     } catch (Exception e) {
         println("waitptext: failed in messge receive");
         return ShellApp.appMiscError;
     }
      return ShellApp.appNoError;
 }

That's all there is to waitptext! Now you can use the makeit.bat file to compile this new command.

Back to top

Testing the waitptext command

Start up both node1 (under the shell directory) and node2 (under the shell2 directory) shell instances. In both instances, type in the following command:

jxta> search -f

This will flush all the existing advertisements that may be around. Because JXTA nodes should survive reboot, advertisements are persistent by default. Using search -f will ensure that we have no leftover advertisements from prior experimentation.

We can now enter our new command on node1. You should see the following output:

JXTA>waitptext dwTag
created a pipe advertisement...
created an input pipe based on the advertisement...
published the pipe advertisement to the group...
waiting at the input pipe for a message...

At this point, the shell instance on node1 is waiting for incoming messages.

On the node2 instance, repeat the steps we took earlier in our shell experimentation to:

1. Find the published pipe advertisement (search -r)
2. Make an output pipe based on the advertisement (dwoutpipe = mkpipe -o adv0)
3. Create a message (dwMsg = mkmsg)
4. Load the text file into a data variable (importfile -f simple.txt dwData)
5. Attach the text file as an element called dwTag (put dwMsg dwTag dwData)
6. Send the message through the pipe (send dwMsg dwoutpipe)

Back on node1, we should see the content of the file sent:

tagName used is dwTag
received a message...
<?xml version="1.0"?>
<ShellDoc>
     <Item>
          This is a simple file for JXTA transfer. 
           It can contain anything.
     </Item>
</ShellDoc>
JXTA>

We have completed the creation of a complex shell extension and have discovered how to create shell extensions that make use of the JXTA platform capabilities.

Back to top

A word on interoperability

Because all of our examples have been based on the Java platform, some readers may be questioning the programming language/platform neutrality of JXTA, a feature we emphasized in the first article of this series. Bear in mind, however, that what we are working with here is a Java implementation of the JXTA core (the reference implementation of JXTA). Consider, in addition, that the JXTA shell is actually a Java-based JXTA application that works on top of the JXTA core, and you can see that we have a very Java-centric environment. But other implementations of both the platform and the shell are possible.

Underneath the hood, the protocol interactions that do the real work are completely interoperable. For example: to join a peergroup, we need to contact the membership peergroup service using the JXTA Peer Membership protocol. To talk to a pipe, we had to use a JXTA Pipe Binding protocol to contact a pipe service. To share an advertisement, we had to use a JXTA Discovery protocol to publish it on the peergroup. You could build an implementation of JXTA and the JXTA shell on another platform that would readily interoperate with our JXTA shell application, assuming it implemented the basic protocols as described in the JXTA protocol specification. (For more on those protocols, check out the first article of this series.)

Back to top

Coming next: Reaching out to the network

The JXTA shell is a great tool for experimenting and learning P2P fundamentals. Its simple yet elegant extensibility makes it easy to add new functionality. In this article, we have worked extensively with the command line of the shell and created our own complex shell extension from scratch.

For the most part, we have only worked with machines within one peergroup, residing on the same local area network. (In fact, we've mostly worked with only one machine). Recall from the first article of this series that a JXTA peergroup boundary is not limited by an underlying physical network topology. In the final article in this series, we will see first-hand how JXTA discovery can be extended to wide area networks (via routers) and across firewalls over the Internet (via a rendezvous service).

Back to top

Download

discuss this topic to forum