Maven dependencies

For your application module

<dependency>
    <groupId>org.apache.twill</groupId>
    <artifactId>twill-api</artifactId>
    <version>0.11.0</version>
</dependency>

For the your Apache Twill YARN client module

<dependency>
    <groupId>org.apache.twill</groupId>
    <artifactId>twill-yarn</artifactId>
    <version>0.11.0</version>
</dependency>

Clone and build the Twill library

You can also clone the source and build the latest snapshot.

$ git clone https://git-wip-us.apache.org/repos/asf/twill.git twill
$ cd twill
$ mvn clean install

Quick example

Let’s begin by building a basic EchoServer in Twill. Traditionally, when you build a server as simple as this, you add logic within a Runnable implementation to run it in a Thread using an appropriate ExecutorService:

public class EchoServer implements Runnable {
  private static Logger LOG = LoggerFactory.getLogger(EchoServer.class);
  private final ServerSocket serverSocket;

  public EchoServer() {
    ...
  }

  @Override
  public void run() {
    while ( isRunning() ) {
      Socket socket = serverSocket.accept();
      ...
    }
  }
}

Our example defines an implementation of Runnable that implements the run() method. The EchoServer is now a Runnable that can be executed by an ExecutorService:

ExecutorService service = Executors.newFixedThreadPool(2);
service.submit(new EchoServer());

The above model is familiar, but now assume you want to run your EchoServer on a YARN cluster. To do this, all you need to do is implement the TwillRunnable interface similarly to how you normally implement Runnable:

public class EchoServer implements TwillRunnable {

  private static Logger LOG = LoggerFactory.getLogger(EchoServer.class);
  private final ServerSocket serverSocket;
  private final int port;

  public EchoServer() {
    ...
  }

  @Override
  public void run() {
    while ( isRunning() ) {
      Socket socket = serverSocket.accept();
      ...
    }
  }
}

To run EchoServer on the YARN cluster, you need a TwillRunnerService, which is similar to ExecutorService. You can specify the YARN cluster configuration and connection string to a running instance of a Zookeeper service to create an instance of TwillRunnerService:

TwillRunnerService runnerService = new YarnTwillRunnerService(
  new YarnConfiguration(), zkConnectStr);
runnerService.start();

Now you are read to run the EchoServer on YARN, by simply preparing and starting through the TwillRunnerService. You can also attach log handler to receives logs generated by EchoService running on some nodes in the cluster:

TwillController controller = runnerService.prepare(new EchoServer())
  .addLogHandler(new PrinterLogHandler(new PrintWriter(System.out)))
  .start();

Now that the EchoServer is started and you can inspect or control the application through the TwillController. For example, you can attach listeners for state changes:

controller.addListener(new ListenerAdapter() {
  @Override
  public void running() {
    LOG.info('Echo Server Started');
  }
}

You can also stop the application through the TwillController:

controller.terminate().get();

This will shut down the application master and all the containers.

Hello World

To see Twill in action, you can run the “hello world” example applications located in the twill-examples module.

$ export CP=twill-examples-yarn-0.10.0.jar:`hadoop classpath`
$ java -cp $CP org.apache.twill.example.yarn.HelloWorld {zookeeper_host:port}

14:49:45.944 [TwillContainerService] INFO  o.a.twill.example.yarn.HelloWorld - Hello World. My first distributed application.

$ export CP=twill-examples-yarn-0.10.0.jar:`hadoop classpath`
$ java -cp $CP org.apache.twill.example.yarn.BundledJarExample {zookeeper_host:port} \
    twill-examples-echo-0.10.0.jar echo.EchoMain arg1

[TwillContainerService] INFO echo.EchoMain - Hello from EchoMain: 6
err HELLO from scatch
[TwillContainerService] INFO echo.EchoMain - Got args: [arg1]

...

out HELLO from scatch
Got args: [arg1]