Start line:  
End line:  

Snippet Preview

Snippet HTML Code

Stack Overflow Questions
Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
 
 package org.apache.hadoop.hbase.snapshot;
 
 
Utility class to help manage SnapshotDesriptions.

Snapshots are laid out on disk like this:

 /hbase/.snapshots
          /.tmp                <---- working directory
          /[snapshot name]     <----- completed snapshot
 
A completed snapshot named 'completed' then looks like (multiple regions, servers, files, etc. signified by '...' on the same directory depth).
 /hbase/.snapshots/completed
                   .snapshotinfo          <--- Description of the snapshot
                   .tableinfo             <--- Copy of the tableinfo
                    /.logs
                        /[server_name]
                            /... [log files]
                         ...
                   /[region name]           <---- All the region's information
                   .regioninfo              <---- Copy of the HRegionInfo
                      /[column family name]
                          /[hfile name]     <--- name of the hfile in the real region
                          ...
                      ...
                    ...
 
Utility methods in this class are useful for getting the correct locations for different parts of the snapshot, as well as moving completed snapshots into place (see completeSnapshot(org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.SnapshotDescription,org.apache.hadoop.fs.Path,org.apache.hadoop.fs.Path,org.apache.hadoop.fs.FileSystem), and writing the org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.SnapshotDescription to the working snapshot directory.
 
 public class SnapshotDescriptionUtils {

  
Filter that only accepts completed snapshot directories
 
   public static class CompletedSnaphotDirectoriesFilter extends FSUtils.BlackListDirFilter {

    

Parameters:
fs
 
       super(fs, Collections.singletonList());
     }
   }
 
   private static final Log LOG = LogFactory.getLog(SnapshotDescriptionUtils.class);

  
Version of the fs layout for new snapshot. Each version of hbase should be able to read older formats.
 
   public static final String SNAPSHOT_LAYOUT_CONF_KEY = "hbase.snapshot.format.version";
 
   // snapshot directory constants
  
The file contains the snapshot basic information and it is under the directory of a snapshot.
  public static final String SNAPSHOTINFO_FILE = ".snapshotinfo";

  
Temporary directory under the snapshot directory to store in-progress snapshots
  public static final String SNAPSHOT_TMP_DIR_NAME = ".tmp";
  // snapshot operation values
  
Default value if no start time is specified
  public static final long NO_SNAPSHOT_START_TIME_SPECIFIED = 0;
  public static final String MASTER_SNAPSHOT_TIMEOUT_MILLIS = "hbase.snapshot.master.timeout.millis";

  
By default, wait 60 seconds for a snapshot to complete
  public static final long DEFAULT_MAX_WAIT_TIME = 60000;
  private SnapshotDescriptionUtils() {
    // private constructor for utility class
  }
  public static int getDefaultSnapshotLayoutFormat(final Configuration conf) {
    if (layoutFormat >= .) {
    }
  }


  

Parameters:
conf org.apache.hadoop.conf.Configuration from which to check for the timeout
type type of snapshot being taken
defaultMaxWaitTime Default amount of time to wait, if none is in the configuration
Returns:
the max amount of time the master should wait for a snapshot to complete
  public static long getMaxMasterTimeout(Configuration confSnapshotDescription.Type type,
      long defaultMaxWaitTime) {
    String confKey;
    switch (type) {
    case :
    default:
      confKey = ;
    }
    return conf.getLong(confKeydefaultMaxWaitTime);
  }

  
Get the snapshot root directory. All the snapshots are kept under this directory, i.e. ${hbase.rootdir}/.snapshot

Parameters:
rootDir hbase root directory
Returns:
the base directory in which all snapshots are kept
  public static Path getSnapshotRootDir(final Path rootDir) {
    return new Path(rootDir.);
  }

  
Get the directory for a specified snapshot. This directory is a sub-directory of snapshot root directory and all the data files for a snapshot are kept under this directory.

Parameters:
snapshot snapshot being taken
rootDir hbase root directory
Returns:
the final directory for the completed snapshot
  public static Path getCompletedSnapshotDir(final SnapshotDescription snapshotfinal Path rootDir) {
    return getCompletedSnapshotDir(snapshot.getName(), rootDir);
  }

  
Get the directory for a completed snapshot. This directory is a sub-directory of snapshot root directory and all the data files for a snapshot are kept under this directory.

Parameters:
snapshotName name of the snapshot being taken
rootDir hbase root directory
Returns:
the final directory for the completed snapshot
  public static Path getCompletedSnapshotDir(final String snapshotNamefinal Path rootDir) {
    return getCompletedSnapshotDir(getSnapshotsDir(rootDir), snapshotName);
  }

  
Get the general working directory for snapshots - where they are built, where they are temporarily copied on export, etc.

Parameters:
rootDir root directory of the HBase installation
Returns:
Path to the snapshot tmp directory, relative to the passed root directory
  public static Path getWorkingSnapshotDir(final Path rootDir) {
    return new Path(getSnapshotsDir(rootDir), );
  }

  
Get the directory to build a snapshot, before it is finalized

Parameters:
snapshot snapshot that will be built
rootDir root directory of the hbase installation
Returns:
org.apache.hadoop.fs.Path where one can build a snapshot
  public static Path getWorkingSnapshotDir(SnapshotDescription snapshotfinal Path rootDir) {
    return getCompletedSnapshotDir(getWorkingSnapshotDir(rootDir), snapshot.getName());
  }

  
Get the directory to build a snapshot, before it is finalized

Parameters:
snapshotName name of the snapshot
rootDir root directory of the hbase installation
Returns:
org.apache.hadoop.fs.Path where one can build a snapshot
  public static Path getWorkingSnapshotDir(String snapshotNamefinal Path rootDir) {
    return getCompletedSnapshotDir(getWorkingSnapshotDir(rootDir), snapshotName);
  }

  
Get the directory to store the snapshot instance

Parameters:
snapshotsDir hbase-global directory for storing all snapshots
snapshotName name of the snapshot to take
Returns:
the final directory for the completed snapshot
  private static final Path getCompletedSnapshotDir(final Path snapshotsDirString snapshotName) {
    return new Path(snapshotsDirsnapshotName);
  }

  

Parameters:
rootDir hbase root directory
Returns:
the directory for all completed snapshots;
  public static final Path getSnapshotsDir(Path rootDir) {
    return new Path(rootDir.);
  }

  
Convert the passed snapshot description into a 'full' snapshot description based on default parameters, if none have been supplied. This resolves any 'optional' parameters that aren't supplied to their default values.

Parameters:
snapshot general snapshot descriptor
conf Configuration to read configured snapshot defaults if snapshot is not complete
Returns:
a valid snapshot description
Throws:
java.lang.IllegalArgumentException if the org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.SnapshotDescription is not a complete org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.SnapshotDescription.
  public static SnapshotDescription validate(SnapshotDescription snapshotConfiguration conf)
      throws IllegalArgumentException {
    if (!snapshot.hasTable()) {
      throw new IllegalArgumentException(
        "Descriptor doesn't apply to a table, so we can't build it.");
    }
    // set the creation time, if one hasn't been set
    long time = snapshot.getCreationTime();
      time = EnvironmentEdgeManager.currentTimeMillis();
      .debug("Creation time not specified, setting to:" + time + " (current time:"
          + EnvironmentEdgeManager.currentTimeMillis() + ").");
      SnapshotDescription.Builder builder = snapshot.toBuilder();
      builder.setCreationTime(time);
      snapshot = builder.build();
    }
    return snapshot;
  }

  
Write the snapshot description into the working directory of a snapshot

Parameters:
snapshot description of the snapshot being taken
workingDir working directory of the snapshot
fs org.apache.hadoop.fs.FileSystem on which the snapshot should be taken
Throws:
java.io.IOException if we can't reach the filesystem and the file cannot be cleaned up on failure
  public static void writeSnapshotInfo(SnapshotDescription snapshotPath workingDirFileSystem fs)
      throws IOException {
    FsPermission perms = FSUtils.getFilePermissions(fsfs.getConf(),
    Path snapshotInfo = new Path(workingDir.);
    try {
      FSDataOutputStream out = FSUtils.create(fssnapshotInfopermstrue);
      try {
        snapshot.writeTo(out);
      } finally {
        out.close();
      }
    } catch (IOException e) {
      // if we get an exception, try to remove the snapshot info
      if (!fs.delete(snapshotInfofalse)) {
        String msg = "Couldn't delete snapshot info file: " + snapshotInfo;
        .error(msg);
        throw new IOException(msg);
      }
    }
  }

  
Read in the org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.SnapshotDescription stored for the snapshot in the passed directory

Parameters:
fs filesystem where the snapshot was taken
snapshotDir directory where the snapshot was stored
Returns:
the stored snapshot description
Throws:
org.apache.hadoop.hbase.snapshot.CorruptedSnapshotException if the snapshot cannot be read
  public static SnapshotDescription readSnapshotInfo(FileSystem fsPath snapshotDir)
      throws CorruptedSnapshotException {
    Path snapshotInfo = new Path(snapshotDir);
    try {
      FSDataInputStream in = null;
      try {
        in = fs.open(snapshotInfo);
        SnapshotDescription desc = SnapshotDescription.parseFrom(in);
        return desc;
      } finally {
        if (in != nullin.close();
      }
    } catch (IOException e) {
      throw new CorruptedSnapshotException("Couldn't read snapshot info from:" + snapshotInfoe);
    }
  }

  
Move the finished snapshot to its final, publicly visible directory - this marks the snapshot as 'complete'.

Parameters:
snapshot description of the snapshot being tabken
rootdir root directory of the hbase installation
workingDir directory where the in progress snapshot was built
fs org.apache.hadoop.fs.FileSystem where the snapshot was built
Throws:
org.apache.hadoop.hbase.snapshot.SnapshotCreationException if the snapshot could not be moved
java.io.IOException the filesystem could not be reached
  public static void completeSnapshot(SnapshotDescription snapshotPath rootdirPath workingDir,
      FileSystem fsthrows SnapshotCreationExceptionIOException {
    Path finishedDir = getCompletedSnapshotDir(snapshotrootdir);
    .debug("Snapshot is done, just moving the snapshot from " + workingDir + " to "
        + finishedDir);
    if (!fs.rename(workingDirfinishedDir)) {
      throw new SnapshotCreationException("Failed to move working directory(" + workingDir
          + ") to completed directory(" + finishedDir + ")."snapshot);
    }
  }
New to GrepCode? Check out our FAQ X