public class RemoveDirectory extends Engine
CreateDirectory
Modifier and Type | Field and Description |
---|---|
static String |
CLEANUP_DAG_PREFIX
The prefix that is attached to the name of the dag for which the
cleanup Dag is being generated, to generate the name of the cleanup
Dag.
|
static String |
CLEANUP_PREFIX
Constant suffix for the names of the deployment nodes.
|
static String |
DERIVATION_NAME
The logical name of the transformation that removes directories on the
remote execution pools.
|
static String |
DERIVATION_NAMESPACE
The derivation namespace for the create dir jobs.
|
static String |
DERIVATION_VERSION
The version number for the derivations for create dir jobs.
|
private ADag |
mConcDag
The concrete dag so far, for which the clean up dag needs to be generated.
|
protected String |
mJobPrefix
The job prefix that needs to be applied to the job file basenames.
|
private String |
mSubmitDirectory
The submit directory for the workflow.
|
private boolean |
mTransferFromSubmitHost
Boolean indicating whether we need to transfer dirmanager from the submit
host.
|
static String |
REMOVE_DIR_EXECUTABLE_BASENAME
The basename of the pegasus dirmanager executable.
|
static String |
REMOVE_DIR_SUFFIX
Constant suffix for the names of the remote directory nodes.
|
static String |
TRANSFORMATION_NAME
The logical name of the transformation that removes directories on the
remote execution pools.
|
static String |
TRANSFORMATION_NAMESPACE
The transformation namespace for the create dir jobs.
|
static String |
TRANSFORMATION_VERSION
The version number for the derivations for create dir jobs.
|
mBag, mLogger, mLogMsg, mOutputPool, mPoolFile, mPOptions, mProps, mRLIUrl, mSiteStore, mTCFile, mTCHandle, mTCMode, REGISTRATION_UNIVERSE, TRANSFER_UNIVERSE
Constructor and Description |
---|
RemoveDirectory(ADag concDag,
PegasusBag bag,
String submitDirectory)
The overloaded constructor that sets the dag for which we have to
generated the cleanup dag for.
|
Modifier and Type | Method and Description |
---|---|
ADag |
addRemoveDirectoryNodes(ADag dag)
Modifies the workflow to add remove directory nodes.
|
ADag |
addRemoveDirectoryNodes(ADag workflow,
Set<String> sites)
Adds create dir nodes to the workflow.
|
private TransformationCatalogEntry |
defaultTCEntry(SiteCatalogEntry site)
Returns a default TC entry to be used in case entry is not found in the
transformation catalog.
|
private String |
getAssociatedCreateDirSite(Job job)
Returns the associated site that job is dependant on.
|
static String |
getCompleteTranformationName()
A convenience method to return the complete transformation name being
used to construct jobs in this class.
|
protected Set |
getCreateDirSites(ADag dag)
Retrieves the sites for which the create dir jobs need to be created.
|
private String |
getRemoveDirJobName(ADag dag,
String site)
It returns the name of the remove directory job, that is to be assigned.
|
Job |
makeRemoveDirJob(String site,
String jobName)
It creates a remove directory job that creates a directory on the remote pool
using the perl executable that Gaurang wrote.
|
Job |
makeRemoveDirJob(String site,
String jobName,
List<String> files)
It creates a remove directory job that creates a directory on the remote pool
using the perl executable that Gaurang wrote.
|
addVector, appendArrayList, complainForHeadNodeURLPrefix, complainForHeadNodeURLPrefix, loadProperties, printVector, stringInList, stringInPegVector, stringInVector, vectorToString
public static final String CLEANUP_DAG_PREFIX
public static final String REMOVE_DIR_SUFFIX
public static final String TRANSFORMATION_NAME
public static final String REMOVE_DIR_EXECUTABLE_BASENAME
public static final String TRANSFORMATION_NAMESPACE
public static final String TRANSFORMATION_VERSION
public static final String DERIVATION_NAMESPACE
public static final String DERIVATION_NAME
public static final String DERIVATION_VERSION
public static final String CLEANUP_PREFIX
private ADag mConcDag
private boolean mTransferFromSubmitHost
private String mSubmitDirectory
protected String mJobPrefix
public RemoveDirectory(ADag concDag, PegasusBag bag, String submitDirectory)
concDag
- the concrete dag for which cleanup is reqd.bag
- the bag of initialization objectssubmitDirectory
- the submit directory for the cleanup workflowpublic static String getCompleteTranformationName()
public ADag addRemoveDirectoryNodes(ADag dag)
dag
- the workflow to which the nodes have to be added.public ADag addRemoveDirectoryNodes(ADag workflow, Set<String> sites)
workflow
- the workflowsites
- the staging sites the workflow refers to.protected Set getCreateDirSites(ADag dag)
private String getRemoveDirJobName(ADag dag, String site)
dag
- the dag for which the cleanup DAG is being generated.site
- the execution site for which the remove directory job
is responsible.public Job makeRemoveDirJob(String site, String jobName)
site
- the execution pool for which the create dir job is to be
created.jobName
- the name that is to be assigned to the job.public Job makeRemoveDirJob(String site, String jobName, List<String> files)
site
- the site from where the directory need to be removed.jobName
- the name that is to be assigned to the job.files
- the list of files to be cleaned up.private TransformationCatalogEntry defaultTCEntry(SiteCatalogEntry site)
site
- the SiteCatalogEntry for the site for which the default entry is required.private String getAssociatedCreateDirSite(Job job)
job
- the job for which we need the associated create dir site.