Start line:  
End line:  

Snippet Preview

Snippet HTML Code

Stack Overflow Questions
  /*
   * Copyright 2010 Amazon.com, Inc. or its affiliates. All Rights Reserved.
   * 
   * Licensed under the Apache License, Version 2.0 (the "License").
   * You may not use this file except in compliance with the License.
   * A copy of the License is located at
   * 
   *  http://aws.amazon.com/apache2.0
   * 
  * or in the "license" file accompanying this file. This file is distributed
  * on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
  * express or implied. See the License for the specific language governing
  * permissions and limitations under the License.
  */
 package com.amazonaws.services.elasticmapreduce;
 
 
 import java.util.List;
 
 
Client for accessing AmazonElasticMapReduce. All service calls made using this client are blocking, and will not return until the service call completes.

This is the Amazon Elastic MapReduce API Reference Guide. This guide is for programmers who need detailed information about the Amazon Elastic MapReduce APIs.

 
 public class AmazonElasticMapReduceClient extends AmazonWebServiceClient implements AmazonElasticMapReduce {

    
The AWS credentials (access key ID and secret key) to use when authenticating with AWS services.
 
     private AWSCredentials awsCredentials;

    
List of exception unmarshallers for all AmazonElasticMapReduce exceptions.
 
     protected final List<Unmarshaller<AmazonServiceExceptionNode>> exceptionUnmarshallers;

    
Low level client for sending requests to AWS services.
 
     protected final HttpClient client;

    
Optional request handlers for additional request processing.
 
     private List<RequestHandlerrequestHandlers = new ArrayList<RequestHandler>();
    
    
AWS signer for authenticating requests.
 
     private QueryStringSigner signer;


    
Constructs a new client to invoke service methods on AmazonElasticMapReduce using the specified AWS account credentials.

All service calls made using this new client object are blocking, and will not return until the service call completes.

Parameters:
awsCredentials The AWS credentials (access key ID and secret key) to use when authenticating with AWS services.
 
     public AmazonElasticMapReduceClient(AWSCredentials awsCredentials) {
         this(awsCredentialsnew ClientConfiguration());
     }

    
Constructs a new client to invoke service methods on AmazonElasticMapReduce using the specified AWS account credentials and client configuration options.

All service calls made using this new client object are blocking, and will not return until the service call completes.

Parameters:
awsCredentials The AWS credentials (access key ID and secret key) to use when authenticating with AWS services.
clientConfiguration The client configuration options controlling how this client connects to AmazonElasticMapReduce (ex: proxy settings, retry counts, etc.).
    public AmazonElasticMapReduceClient(AWSCredentials awsCredentialsClientConfiguration clientConfiguration) {
        super(clientConfiguration);
        this. = awsCredentials;
        
        setEndpoint("elasticmapreduce.amazonaws.com");
         = new QueryStringSigner(awsCredentials);
                "/com/amazonaws/services/elasticmapreduce/request.handlers");
         = new HttpClient(clientConfiguration);
    }

    
    

AddInstanceGroups adds an instance group to a running cluster.

Parameters:
addInstanceGroupsRequest Container for the necessary parameters to execute the AddInstanceGroups service method on AmazonElasticMapReduce.
Returns:
The response from the AddInstanceGroups service method, as returned by AmazonElasticMapReduce.
Throws:
com.amazonaws.services.elasticmapreduce.model.InternalServerErrorException
com.amazonaws.AmazonClientException If any internal errors are encountered inside the client while attempting to make the request or handle the response. For example if a network connection is not available.
com.amazonaws.AmazonServiceException If an error response is returned by AmazonElasticMapReduce indicating either a problem with the data in the request, or a server side issue.
    public AddInstanceGroupsResult addInstanceGroups(AddInstanceGroupsRequest addInstanceGroupsRequest
            throws AmazonServiceExceptionAmazonClientException {
        Request<AddInstanceGroupsRequestrequest = new AddInstanceGroupsRequestMarshaller().marshall(addInstanceGroupsRequest);
        return invoke(requestnew AddInstanceGroupsResultStaxUnmarshaller());
    }
    
    

AddJobFlowSteps adds new steps to a running job flow. A maximum of 256 steps are allowed in each job flow.

A step specifies the location of a JAR file stored either on the master node of the job flow or in Amazon S3. Each step is performed by the main function of the main class of the JAR file. The main class can be specified either in the manifest of the JAR or by using the MainFunction parameter of the step.

Elastic MapReduce executes each step in the order listed. For a step to be considered complete, the main function must exit with a zero exit code and all Hadoop jobs started while the step was running must have completed and run successfully.

You can only add steps to a job flow that is in one of the following states: STARTING, BOOTSTAPPING, RUNNING, or WAITING.

Parameters:
addJobFlowStepsRequest Container for the necessary parameters to execute the AddJobFlowSteps service method on AmazonElasticMapReduce.
Throws:
com.amazonaws.services.elasticmapreduce.model.InternalServerErrorException
com.amazonaws.AmazonClientException If any internal errors are encountered inside the client while attempting to make the request or handle the response. For example if a network connection is not available.
com.amazonaws.AmazonServiceException If an error response is returned by AmazonElasticMapReduce indicating either a problem with the data in the request, or a server side issue.
    public void addJobFlowSteps(AddJobFlowStepsRequest addJobFlowStepsRequest
            throws AmazonServiceExceptionAmazonClientException {
        Request<AddJobFlowStepsRequestrequest = new AddJobFlowStepsRequestMarshaller().marshall(addJobFlowStepsRequest);
        invoke(requestnull);
    }
    
    

TerminateJobFlows shuts a list of job flows down. When a job flow is shut down, any step not yet completed is canceled and the EC2 instances on which the job flow is running are stopped. Any log files not already saved are uploaded to Amazon S3 if a LogUri was specified when the job flow was created.

Parameters:
terminateJobFlowsRequest Container for the necessary parameters to execute the TerminateJobFlows service method on AmazonElasticMapReduce.
Throws:
com.amazonaws.services.elasticmapreduce.model.InternalServerErrorException
com.amazonaws.AmazonClientException If any internal errors are encountered inside the client while attempting to make the request or handle the response. For example if a network connection is not available.
com.amazonaws.AmazonServiceException If an error response is returned by AmazonElasticMapReduce indicating either a problem with the data in the request, or a server side issue.
    public void terminateJobFlows(TerminateJobFlowsRequest terminateJobFlowsRequest
            throws AmazonServiceExceptionAmazonClientException {
        Request<TerminateJobFlowsRequestrequest = new TerminateJobFlowsRequestMarshaller().marshall(terminateJobFlowsRequest);
        invoke(requestnull);
    }
    
    

DescribeJobFlows returns a list of job flows that match all of the supplied parameters. The parameters can include a list of job flow IDs, job flow states, and restrictions on job flow creation date and time.

Regardless of supplied parameters, only job flows created within the last two months are returned.

If no parameters are supplied, then job flows matching either of the following criteria are returned:

  • Job flows created and completed in the last two weeks
  • Job flows created within the last two months that are in one of the following states: RUNNING , WAITING , SHUTTING_DOWN , STARTING

Amazon Elastic MapReduce can return a maximum of 512 job flow descriptions.

Parameters:
describeJobFlowsRequest Container for the necessary parameters to execute the DescribeJobFlows service method on AmazonElasticMapReduce.
Returns:
The response from the DescribeJobFlows service method, as returned by AmazonElasticMapReduce.
Throws:
com.amazonaws.services.elasticmapreduce.model.InternalServerErrorException
com.amazonaws.AmazonClientException If any internal errors are encountered inside the client while attempting to make the request or handle the response. For example if a network connection is not available.
com.amazonaws.AmazonServiceException If an error response is returned by AmazonElasticMapReduce indicating either a problem with the data in the request, or a server side issue.
    public DescribeJobFlowsResult describeJobFlows(DescribeJobFlowsRequest describeJobFlowsRequest
            throws AmazonServiceExceptionAmazonClientException {
        Request<DescribeJobFlowsRequestrequest = new DescribeJobFlowsRequestMarshaller().marshall(describeJobFlowsRequest);
        return invoke(requestnew DescribeJobFlowsResultStaxUnmarshaller());
    }
    
    

RunJobFlow creates and starts running a new job flow. The job flow will run the steps specified. Once the job flow completes, the cluster is stopped and the HDFS partition is lost. To prevent loss of data, configure the last step of the job flow to store results in Amazon S3. If the JobFlowInstancesDetail : KeepJobFlowAliveWhenNoSteps parameter is set to TRUE , the job flow will transition to the WAITING state rather than shutting down once the steps have completed.

A maximum of 256 steps are allowed in each job flow.

For long running job flows, we recommended that you periodically store your results.

Parameters:
runJobFlowRequest Container for the necessary parameters to execute the RunJobFlow service method on AmazonElasticMapReduce.
Returns:
The response from the RunJobFlow service method, as returned by AmazonElasticMapReduce.
Throws:
com.amazonaws.services.elasticmapreduce.model.InternalServerErrorException
com.amazonaws.AmazonClientException If any internal errors are encountered inside the client while attempting to make the request or handle the response. For example if a network connection is not available.
com.amazonaws.AmazonServiceException If an error response is returned by AmazonElasticMapReduce indicating either a problem with the data in the request, or a server side issue.
    public RunJobFlowResult runJobFlow(RunJobFlowRequest runJobFlowRequest
            throws AmazonServiceExceptionAmazonClientException {
        Request<RunJobFlowRequestrequest = new RunJobFlowRequestMarshaller().marshall(runJobFlowRequest);
        return invoke(requestnew RunJobFlowResultStaxUnmarshaller());
    }
    
    

ModifyInstanceGroups modifies the number of nodes and configuration settings of an instance group. The input parameters include the new target instance count for the group and the instance group ID. The call will either succeed or fail atomically.

Parameters:
modifyInstanceGroupsRequest Container for the necessary parameters to execute the ModifyInstanceGroups service method on AmazonElasticMapReduce.
Throws:
com.amazonaws.services.elasticmapreduce.model.InternalServerErrorException
com.amazonaws.AmazonClientException If any internal errors are encountered inside the client while attempting to make the request or handle the response. For example if a network connection is not available.
com.amazonaws.AmazonServiceException If an error response is returned by AmazonElasticMapReduce indicating either a problem with the data in the request, or a server side issue.
    public void modifyInstanceGroups(ModifyInstanceGroupsRequest modifyInstanceGroupsRequest
            throws AmazonServiceExceptionAmazonClientException {
        Request<ModifyInstanceGroupsRequestrequest = new ModifyInstanceGroupsRequestMarshaller().marshall(modifyInstanceGroupsRequest);
        invoke(requestnull);
    }
    
    

DescribeJobFlows returns a list of job flows that match all of the supplied parameters. The parameters can include a list of job flow IDs, job flow states, and restrictions on job flow creation date and time.

Regardless of supplied parameters, only job flows created within the last two months are returned.

If no parameters are supplied, then job flows matching either of the following criteria are returned:

  • Job flows created and completed in the last two weeks
  • Job flows created within the last two months that are in one of the following states: RUNNING , WAITING , SHUTTING_DOWN , STARTING

Amazon Elastic MapReduce can return a maximum of 512 job flow descriptions.

Returns:
The response from the DescribeJobFlows service method, as returned by AmazonElasticMapReduce.
Throws:
com.amazonaws.services.elasticmapreduce.model.InternalServerErrorException
com.amazonaws.AmazonClientException If any internal errors are encountered inside the client while attempting to make the request or handle the response. For example if a network connection is not available.
com.amazonaws.AmazonServiceException If an error response is returned by AmazonElasticMapReduce indicating either a problem with the data in the request, or a server side issue.
        return describeJobFlows(new DescribeJobFlowsRequest());
    }
    
    

ModifyInstanceGroups modifies the number of nodes and configuration settings of an instance group. The input parameters include the new target instance count for the group and the instance group ID. The call will either succeed or fail atomically.

Throws:
com.amazonaws.services.elasticmapreduce.model.InternalServerErrorException
com.amazonaws.AmazonClientException If any internal errors are encountered inside the client while attempting to make the request or handle the response. For example if a network connection is not available.
com.amazonaws.AmazonServiceException If an error response is returned by AmazonElasticMapReduce indicating either a problem with the data in the request, or a server side issue.
    }
    


    
Returns additional metadata for a previously executed successful, request, typically used for debugging issues where a service isn't acting as expected. This data isn't considered part of the result data returned by an operation, so it's available through this separate, diagnostic interface.

Response metadata is only cached for a limited period of time, so if you need to access this extra diagnostic information for an executed request, you should use this method to retrieve it as soon as possible after executing the request.

Parameters:
request The originally executed request
Returns:
The response metadata for the specified request, or null if none is available.
        return .getResponseMetadataForRequest(request);
    }
    private <X, Y extends AmazonWebServiceRequest> X invoke(Request<Y> requestUnmarshaller<X, StaxUnmarshallerContextunmarshaller) {
        request.setEndpoint();
        for (Entry<StringStringentry : request.getOriginalRequest().copyPrivateRequestParameters().entrySet()) {
            request.addParameter(entry.getKey(), entry.getValue());
        }
        // Apply any additional service specific request handlers that need to be run
        if ( != null) {
            for (RequestHandler requestHandler : ) {
                request = requestHandler.handleRequest(request);
            }
        }
        try {
            .sign(request);
        } catch (SignatureException e) {
            throw new AmazonServiceException("Unable to sign request"e);
        }
        HttpRequest httpRequest = convertToHttpRequest(request.);
        
        StaxResponseHandler<X> responseHandler = new StaxResponseHandler<X>(unmarshaller);
        return (X).execute(httpRequestresponseHandlererrorResponseHandler);
    }
}
        
New to GrepCode? Check out our FAQ X