Start line:  
End line:  

Snippet Preview

Snippet HTML Code

Stack Overflow Questions
   * Licensed to the Apache Software Foundation (ASF) under one
   * or more contributor license agreements.  See the NOTICE file
   * distributed with this work for additional information
   * regarding copyright ownership.  The ASF licenses this file
   * to you under the Apache License, Version 2.0 (the
   * "License"); you may not use this file except in compliance
   * with the License.  You may obtain a copy of the License at
  * Unless required by applicable law or agreed to in writing, software
  * distributed under the License is distributed on an "AS IS" BASIS,
  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  * See the License for the specific language governing permissions and
  * limitations under the License.
 package org.apache.pig;
 import java.util.List;
 import java.util.Map;

Base class for Pig UDFs that are functions from Tuples to generic type OUT. Handles marshalling objects, basic error checking, etc. Also infers outputSchema and provides a function to verify the input Tuple.

Extend this class and implement the
OUT exec(Tuple input)
method when writting a UDF that operates on multiple inputs from the Tuple.
 public abstract class TypedOutputEvalFunc<OUT> extends EvalFunc<OUT> {
     // Used to implement outputSchema below.
     protected Class<OUT> outTypeClass = null;
     public Class<OUT> getOutputTypeClass() { return ; }
     public TypedOutputEvalFunc() {
          = (Class<OUT>) getTypeArguments(TypedOutputEvalFunc.classgetClass()).get(0);
     // Increment Hadoop counters for bad inputs which are either null or too small.
     protected void verifyInput(Tuple inputint minimumSizethrows IOException {
         verifyUdfInput(getCounterGroup(), inputminimumSize);

Incremented counters will use this as the counter group. Typically this works fine, since the subclass name is enough to identify the UDF. In some cases though (i.e. a UDF wrapper that is a facade to a number of different transformation functions), a more specific group name is needed.
     protected String getCounterGroup() {
         return getClass().getName();

Get the actual type arguments a child class has used to extend a generic base class.

baseClass the base class
childClass the child class
a list of the raw classes for the actual type arguments.
     protected static <T> List<Class<?>> getTypeArguments(Class<T> baseClass,
         Class<? extends T> childClass) {
         Map<TypeTyperesolvedTypes = Maps.newHashMap();
         Type type = childClass;
         // start walking up the inheritance hierarchy until we hit baseClass
         while (!getClass(type).equals(baseClass)) {
             if (type instanceof Class) {
                 // there is no useful information for us in raw types, so just keep going.
                 type = ((Classtype).getGenericSuperclass();
             } else {
                 ParameterizedType parameterizedType = (ParameterizedTypetype;
                 Class<?> rawType = (ClassparameterizedType.getRawType();
                 Type[] actualTypeArguments = parameterizedType.getActualTypeArguments();
                 TypeVariable<?>[] typeParameters = rawType.getTypeParameters();
                 for (int i = 0; i < actualTypeArguments.lengthi++) {
                     resolvedTypes.put(typeParameters[i], actualTypeArguments[i]);
                 if (!rawType.equals(baseClass)) {
                     type = rawType.getGenericSuperclass();
         // finally, for each actual type argument provided to baseClass, determine (if possible)
        // the raw class for that type argument.
        Type[] actualTypeArguments;
        if (type instanceof Class) {
            actualTypeArguments = ((Classtype).getTypeParameters();
        } else {
            actualTypeArguments = ((ParameterizedTypetype).getActualTypeArguments();
        List<Class<?>> typeArgumentsAsClasses = new ArrayList<Class<?>>();
        // resolve types by chasing down type variables.
        for (Type baseType : actualTypeArguments) {
            while (resolvedTypes.containsKey(baseType)) {
                baseType = resolvedTypes.get(baseType);
        return typeArgumentsAsClasses;

Get the underlying class for a type, or null if the type is a variable type.

type the type
the underlying class
    private static Class<?> getClass(Type type) {
        if (type instanceof Class) {
            return (Classtype;
        } else if (type instanceof ParameterizedType) {
            return getClass(((ParameterizedTypetype).getRawType());
        } else if (type instanceof GenericArrayType) {
            Type componentType = ((GenericArrayTypetype).getGenericComponentType();
            Class<?> componentClass = getClass(componentType);
            if (componentClass != null) {
                return Array.newInstance(componentClass, 0).getClass();
            } else {
                return null;
        } else {
            return null;

Increment Hadoop counters for bad inputs which are either null or too small.

klass the name of the calling class, for recording purposes
input the tuple passed to the UDF.
minimumSize the minimum size required of the tuple.
    protected static void verifyUdfInput(String klassTuple inputint minimumSizethrows IOException {
        if (input == null) {
            safeIncrCounter(klass"NullInput", 1L);
            throw new IOException("Null input to UDF " + klass);
        } else if (input.size() < minimumSize) {
             String reason = "TooFewArguments_Got_" + input.size() + "_NeededAtLeast_" + minimumSize;
              safeIncrCounter(klassreason, 1L);
            throw new IOException("Not enough arguments to " + klass + ": got " + input.size() +
                    ", expected at least " + minimumSize);
        } else {
            safeIncrCounter(klass"ValidInput", 1L);
    protected static void safeIncrCounter(String groupString nameLong increment) {
        Counter counter = PigStatusReporter.getInstance().getCounter(groupname);
        if (counter != null) {
New to GrepCode? Check out our FAQ X