org.apache.hadoop.mapred
Class TaskController

java.lang.Object
  extended by org.apache.hadoop.mapred.TaskController
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable
Direct Known Subclasses:
DefaultTaskController

public abstract class TaskController
extends java.lang.Object
implements org.apache.hadoop.conf.Configurable

Controls initialization, finalization and clean up of tasks, and also the launching and killing of task JVMs. This class defines the API for initializing, finalizing and cleaning up of tasks, as also the launching and killing task JVMs. Subclasses of this class will implement the logic required for performing the actual actions.
NOTE: This class is internal only class and not intended for users!!


Nested Class Summary
static class TaskController.DeletionContext
           
 
Field Summary
protected  org.apache.hadoop.fs.LocalDirAllocator allocator
           
protected static java.lang.String COMMAND_FILE
           
protected  org.apache.hadoop.mapred.TaskTracker.LocalStorage localStorage
           
static org.apache.commons.logging.Log LOG
           
static org.apache.hadoop.fs.permission.FsPermission TASK_LAUNCH_SCRIPT_PERMISSION
           
 
Constructor Summary
TaskController()
           
 
Method Summary
abstract  void createLogDir(TaskAttemptID taskID, boolean isCleanup)
          Creates task log dir
abstract  void deleteAsUser(java.lang.String user, java.lang.String subDir)
          Delete the user's files under all of the task tracker root directories.
abstract  void deleteLogAsUser(java.lang.String user, java.lang.String subDir)
          Delete the user's files under the userlogs directory.
 org.apache.hadoop.conf.Configuration getConf()
           
 java.lang.String[] getLocalDirs()
           
 java.lang.String getRunAsUser(JobConf conf)
          Returns the local unix user that a given job will run as.
abstract  void initializeJob(java.lang.String user, java.lang.String jobid, org.apache.hadoop.fs.Path credentials, org.apache.hadoop.fs.Path jobConf, TaskUmbilicalProtocol taskTracker, java.net.InetSocketAddress ttAddr)
          Create all of the directories necessary for the job to start and download all of the job and private distributed cache files.
abstract  int launchTask(java.lang.String user, java.lang.String jobId, java.lang.String attemptId, java.util.List<java.lang.String> setup, java.util.List<java.lang.String> jvmArguments, java.io.File currentWorkDirectory, java.lang.String stdout, java.lang.String stderr)
          Create all of the directories for the task and launches the child jvm.
protected  void logOutput(java.lang.String output)
           
 void setConf(org.apache.hadoop.conf.Configuration conf)
           
abstract  void setup(org.apache.hadoop.fs.LocalDirAllocator allocator, org.apache.hadoop.mapred.TaskTracker.LocalStorage localStorage)
          Does initialization and setup.
abstract  void signalTask(java.lang.String user, int taskPid, ProcessTree.Signal signal)
          Send a signal to a task pid as the user.
abstract  void truncateLogsAsUser(java.lang.String user, java.util.List<Task> allAttempts)
          Run the passed command as the user
protected static java.lang.String writeCommand(java.lang.String cmdLine, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path commandFile)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

COMMAND_FILE

protected static final java.lang.String COMMAND_FILE
See Also:
Constant Field Values

allocator

protected org.apache.hadoop.fs.LocalDirAllocator allocator

localStorage

protected org.apache.hadoop.mapred.TaskTracker.LocalStorage localStorage

TASK_LAUNCH_SCRIPT_PERMISSION

public static final org.apache.hadoop.fs.permission.FsPermission TASK_LAUNCH_SCRIPT_PERMISSION
Constructor Detail

TaskController

public TaskController()
Method Detail

getConf

public org.apache.hadoop.conf.Configuration getConf()
Specified by:
getConf in interface org.apache.hadoop.conf.Configurable

getLocalDirs

public java.lang.String[] getLocalDirs()

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)
Specified by:
setConf in interface org.apache.hadoop.conf.Configurable

setup

public abstract void setup(org.apache.hadoop.fs.LocalDirAllocator allocator,
                           org.apache.hadoop.mapred.TaskTracker.LocalStorage localStorage)
                    throws java.io.IOException
Does initialization and setup.

Parameters:
allocator - the local dir allocator to use
localStorage - TaskTracker's LocalStorage object
Throws:
java.io.IOException

initializeJob

public abstract void initializeJob(java.lang.String user,
                                   java.lang.String jobid,
                                   org.apache.hadoop.fs.Path credentials,
                                   org.apache.hadoop.fs.Path jobConf,
                                   TaskUmbilicalProtocol taskTracker,
                                   java.net.InetSocketAddress ttAddr)
                            throws java.io.IOException,
                                   java.lang.InterruptedException
Create all of the directories necessary for the job to start and download all of the job and private distributed cache files. Creates both the user directories and the job log directory.

Parameters:
user - the user name
jobid - the job
credentials - a filename containing the job secrets
jobConf - the path to the localized configuration file
taskTracker - the connection the task tracker
ttAddr - the tasktracker's RPC address
Throws:
java.io.IOException
java.lang.InterruptedException

launchTask

public abstract int launchTask(java.lang.String user,
                               java.lang.String jobId,
                               java.lang.String attemptId,
                               java.util.List<java.lang.String> setup,
                               java.util.List<java.lang.String> jvmArguments,
                               java.io.File currentWorkDirectory,
                               java.lang.String stdout,
                               java.lang.String stderr)
                        throws java.io.IOException
Create all of the directories for the task and launches the child jvm.

Parameters:
user - the user name
jobId - the jobId in question
attemptId - the attempt id (cleanup attempts have .cleanup suffix)
setup - list of shell commands to execute before the jvm
jvmArguments - list of jvm arguments
currentWorkDirectory - the full path of the cwd for the task
stdout - the file to redirect stdout to
stderr - the file to redirect stderr to
Returns:
the exit code for the task
Throws:
java.io.IOException

signalTask

public abstract void signalTask(java.lang.String user,
                                int taskPid,
                                ProcessTree.Signal signal)
                         throws java.io.IOException
Send a signal to a task pid as the user.

Parameters:
user - the user name
taskPid - the pid of the task
signal - the id of the signal to send
Throws:
java.io.IOException

deleteAsUser

public abstract void deleteAsUser(java.lang.String user,
                                  java.lang.String subDir)
                           throws java.io.IOException
Delete the user's files under all of the task tracker root directories.

Parameters:
user - the user name
subDir - the path relative to the user's subdirectory under the task tracker root directories.
Throws:
java.io.IOException

createLogDir

public abstract void createLogDir(TaskAttemptID taskID,
                                  boolean isCleanup)
                           throws java.io.IOException
Creates task log dir

Parameters:
taskID - ID of the task
isCleanup - If the task is cleanup task or not
Throws:
java.io.IOException

deleteLogAsUser

public abstract void deleteLogAsUser(java.lang.String user,
                                     java.lang.String subDir)
                              throws java.io.IOException
Delete the user's files under the userlogs directory.

Parameters:
user - the user to work as
subDir - the path under the userlogs directory.
Throws:
java.io.IOException

truncateLogsAsUser

public abstract void truncateLogsAsUser(java.lang.String user,
                                        java.util.List<Task> allAttempts)
                                 throws java.io.IOException
Run the passed command as the user

Parameters:
user -
allAttempts - the list of attempts that the JVM ran
Throws:
java.io.IOException

getRunAsUser

public java.lang.String getRunAsUser(JobConf conf)
Returns the local unix user that a given job will run as.


writeCommand

protected static java.lang.String writeCommand(java.lang.String cmdLine,
                                               org.apache.hadoop.fs.FileSystem fs,
                                               org.apache.hadoop.fs.Path commandFile)
                                        throws java.io.IOException
Throws:
java.io.IOException

logOutput

protected void logOutput(java.lang.String output)


Copyright © 2009 The Apache Software Foundation