Start line:  
End line:  

Snippet Preview

Snippet HTML Code

Stack Overflow Questions
Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
 
 
 package org.apache.hadoop.hbase.io;
 
 
HFileLink describes a link to an hfile. An hfile can be served from a region or from the hfile archive directory (/hbase/.archive) HFileLink allows to access the referenced hfile regardless of the location where it is.

Searches for hfiles in the following order and locations:

  • /hbase/table/region/cf/hfile
  • /hbase/.archive/table/region/cf/hfile
The link checks first in the original path if it is not present it fallbacks to the archived path.
 
 public class HFileLink extends FileLink {
   private static final Log LOG = LogFactory.getLog(HFileLink.class);

  
A non-capture group, for HFileLink, so that this can be embedded. The HFileLink describe a link to an hfile in a different table/region and the name is in the form: table=region-hfile.

Table name is ([a-zA-Z_0-9][a-zA-Z_0-9.-]*), so '=' is an invalid character for the table name. Region name is ([a-f0-9]+), so '-' is an invalid character for the region name. HFile is ([0-9a-f]+(?:_SeqId_[0-9]+_)?) covering the plain hfiles (uuid) and the bulk loaded (_SeqId_[0-9]+_) hfiles.

 
   public static final String LINK_NAME_REGEX =
     String.format("(?:(?:%s=)?)%s=%s-%s",
Define the HFile Link name parser in the form of: table=region-hfile
 
   //made package private for testing
   static final Pattern LINK_NAME_PATTERN =
     Pattern.compile(String.format("^(?:(%s)(?:\\=))?(%s)=(%s)-(%s)$",
The pattern should be used for hfile and reference links that can be found in /hbase/table/region/family/
 
   private static final Pattern REF_OR_HFILE_LINK_PATTERN =
     Pattern.compile(String.format("^(?:(%s)(?:=))?(%s)=(%s)-(.+)$",
 
   private final Path archivePath;
   private final Path originPath;
   private final Path tempPath;

  

Parameters:
conf org.apache.hadoop.conf.Configuration from which to extract specific archive locations
path The path of the HFile Link.
Throws:
java.io.IOException on unexpected error.
 
   public HFileLink(Configuration confPath paththrows IOException {
    this(FSUtils.getRootDir(conf), HFileArchiveUtil.getArchivePath(conf), path);
  }

  

Parameters:
rootDir Path to the root directory where hbase files are stored
archiveDir Path to the hbase archive directory
path The path of the HFile Link.
  public HFileLink(final Path rootDirfinal Path archiveDirfinal Path path) {
    Path hfilePath = getRelativeTablePath(path);
    this. = new Path(new Path(rootDir.), hfilePath);
    this. = new Path(rootDirhfilePath);
    this. = new Path(archiveDirhfilePath);
  }

  
Create an HFileLink relative path for the table/region/family/hfile location

Parameters:
table Table name
region Region Name
family Family Name
hfile HFile Name
Returns:
the relative Path to open the specified table/region/family/hfile link
  public static Path createPath(final TableName tablefinal String region,
      final String familyfinal String hfile) {
    if (HFileLink.isHFileLink(hfile)) {
      return new Path(familyhfile);
    }
    return new Path(family, HFileLink.createHFileLinkName(tableregionhfile));
  }

  
Create an HFileLink instance from table/region/family/hfile location

Parameters:
conf org.apache.hadoop.conf.Configuration from which to extract specific archive locations
table Table name
region Region Name
family Family Name
hfile HFile Name
Returns:
Link to the file with the specified table/region/family/hfile location
Throws:
java.io.IOException on unexpected error.
  public static HFileLink create(final Configuration conffinal TableName table,
      final String regionfinal String familyfinal String hfilethrows IOException {
    return new HFileLink(confcreatePath(tableregionfamilyhfile));
  }

  

Returns:
the origin path of the hfile.
  public Path getOriginPath() {
    return this.;
  }

  

Returns:
the path of the archived hfile.
  public Path getArchivePath() {
    return this.;
  }

  

Parameters:
path Path to check.
Returns:
True if the path is a HFileLink.
  public static boolean isHFileLink(final Path path) {
    return isHFileLink(path.getName());
  }


  

Parameters:
fileName File name to check.
Returns:
True if the path is a HFileLink.
  public static boolean isHFileLink(String fileName) {
    Matcher m = .matcher(fileName);
    if (!m.matches()) return false;
    return m.groupCount() > 2 && m.group(4) != null && m.group(3) != null && m.group(2) != null;
  }

  
Convert a HFileLink path to a table relative path. e.g. the link: /hbase/test/0123/cf/testtb=4567-abcd becomes: /hbase/testtb/4567/cf/abcd

Parameters:
path HFileLink path
Returns:
Relative table path
Throws:
java.io.IOException on unexpected error.
  private static Path getRelativeTablePath(final Path path) {
    // table=region-hfile
    if (!m.matches()) {
      throw new IllegalArgumentException(path.getName() + " is not a valid HFileLink name!");
    }
    // Convert the HFileLink name into a real table/region/cf/hfile path.
    TableName tableName = TableName.valueOf(m.group(1), m.group(2));
    String regionName = m.group(3);
    String hfileName = m.group(4);
    String familyName = path.getParent().getName();
    Path tableDir = FSUtils.getTableDir(new Path("./"), tableName);
    return new Path(tableDirnew Path(regionNamenew Path(familyName,
        hfileName)));
  }

  
Get the HFile name of the referenced link

Parameters:
fileName HFileLink file name
Returns:
the name of the referenced HFile
  public static String getReferencedHFileName(final String fileName) {
    Matcher m = .matcher(fileName);
    if (!m.matches()) {
      throw new IllegalArgumentException(fileName + " is not a valid HFileLink name!");
    }
    return(m.group(4));
  }

  
Get the Region name of the referenced link

Parameters:
fileName HFileLink file name
Returns:
the name of the referenced Region
  public static String getReferencedRegionName(final String fileName) {
    Matcher m = .matcher(fileName);
    if (!m.matches()) {
      throw new IllegalArgumentException(fileName + " is not a valid HFileLink name!");
    }
    return(m.group(3));
  }

  
Get the Table name of the referenced link

Parameters:
fileName HFileLink file name
Returns:
the name of the referenced Table
  public static TableName getReferencedTableName(final String fileName) {
    Matcher m = .matcher(fileName);
    if (!m.matches()) {
      throw new IllegalArgumentException(fileName + " is not a valid HFileLink name!");
    }
    return(TableName.valueOf(m.group(1), m.group(2)));
  }

  
Create a new HFileLink name

Parameters:
hfileRegionInfo - Linked HFile Region Info
hfileName - Linked HFile name
Returns:
file name of the HFile Link
  public static String createHFileLinkName(final HRegionInfo hfileRegionInfo,
      final String hfileName) {
    return createHFileLinkName(hfileRegionInfo.getTable(),
                      hfileRegionInfo.getEncodedName(), hfileName);
  }

  
Create a new HFileLink name

Parameters:
tableName - Linked HFile table name
regionName - Linked HFile region name
hfileName - Linked HFile name
Returns:
file name of the HFile Link
  public static String createHFileLinkName(final TableName tableName,
      final String regionNamefinal String hfileName) {
    String s = String.format("%s=%s-%s",
        tableName.getNameAsString().replace(.'='),
        regionNamehfileName);
    return s;
  }

  
Create a new HFileLink

It also adds a back-reference to the hfile back-reference directory to simplify the reference-count and the cleaning process.

Parameters:
conf org.apache.hadoop.conf.Configuration to read for the archive directory name
fs org.apache.hadoop.fs.FileSystem on which to write the HFileLink
dstFamilyPath - Destination path (table/region/cf/)
hfileRegionInfo - Linked HFile Region Info
hfileName - Linked HFile name
Returns:
true if the file is created, otherwise the file exists.
Throws:
java.io.IOException on file or parent directory creation failure
  public static boolean create(final Configuration conffinal FileSystem fs,
      final Path dstFamilyPathfinal HRegionInfo hfileRegionInfo,
      final String hfileNamethrows IOException {
    TableName linkedTable = hfileRegionInfo.getTable();
    String linkedRegion = hfileRegionInfo.getEncodedName();
    return create(conffsdstFamilyPathlinkedTablelinkedRegionhfileName);
  }

  
Create a new HFileLink

It also adds a back-reference to the hfile back-reference directory to simplify the reference-count and the cleaning process.

Parameters:
conf org.apache.hadoop.conf.Configuration to read for the archive directory name
fs org.apache.hadoop.fs.FileSystem on which to write the HFileLink
dstFamilyPath - Destination path (table/region/cf/)
linkedTable - Linked Table Name
linkedRegion - Linked Region Name
hfileName - Linked HFile name
Returns:
true if the file is created, otherwise the file exists.
Throws:
java.io.IOException on file or parent directory creation failure
  public static boolean create(final Configuration conffinal FileSystem fs,
      final Path dstFamilyPathfinal TableName linkedTablefinal String linkedRegion,
      final String hfileNamethrows IOException {
    String familyName = dstFamilyPath.getName();
    String regionName = dstFamilyPath.getParent().getName();
    String tableName = FSUtils.getTableName(dstFamilyPath.getParent().getParent())
        .getNameAsString();
    String name = createHFileLinkName(linkedTablelinkedRegionhfileName);
    String refName = createBackReferenceName(tableNameregionName);
    // Make sure the destination directory exists
    fs.mkdirs(dstFamilyPath);
    // Make sure the FileLink reference directory exists
    Path archiveStoreDir = HFileArchiveUtil.getStoreArchivePath(conf,
          linkedTablelinkedRegionfamilyName);
    Path backRefssDir = getBackReferencesDir(archiveStoreDirhfileName);
    fs.mkdirs(backRefssDir);
    // Create the reference for the link
    Path backRefPath = new Path(backRefssDirrefName);
    fs.createNewFile(backRefPath);
    try {
      // Create the link
      return fs.createNewFile(new Path(dstFamilyPathname));
    } catch (IOException e) {
      .error("couldn't create the link=" + name + " for " + dstFamilyPathe);
      // Revert the reference if the link creation failed
      fs.delete(backRefPathfalse);
      throw e;
    }
  }

  
Create a new HFileLink starting from a hfileLink name

It also adds a back-reference to the hfile back-reference directory to simplify the reference-count and the cleaning process.

Parameters:
conf org.apache.hadoop.conf.Configuration to read for the archive directory name
fs org.apache.hadoop.fs.FileSystem on which to write the HFileLink
dstFamilyPath - Destination path (table/region/cf/)
hfileLinkName - HFileLink name (it contains hfile-region-table)
Returns:
true if the file is created, otherwise the file exists.
Throws:
java.io.IOException on file or parent directory creation failure
  public static boolean createFromHFileLink(final Configuration conffinal FileSystem fs,
      final Path dstFamilyPathfinal String hfileLinkNamethrows IOException {
    Matcher m = .matcher(hfileLinkName);
    if (!m.matches()) {
      throw new IllegalArgumentException(hfileLinkName + " is not a valid HFileLink name!");
    }
    return create(conffsdstFamilyPath, TableName.valueOf(m.group(1), m.group(2)),
        m.group(3), m.group(4));
  }

  
Create the back reference name
  //package-private for testing
  static String createBackReferenceName(final String tableNameStr,
                                        final String regionName) {
    return regionName + "." + tableNameStr.replace(.'=');
  }

  
Get the full path of the HFile referenced by the back reference

Parameters:
rootDir root hbase directory
linkRefPath Link Back Reference path
Returns:
full path of the referenced hfile
Throws:
java.io.IOException on unexpected error.
  public static Path getHFileFromBackReference(final Path rootDirfinal Path linkRefPath) {
    Pair<TableNameStringp = parseBackReferenceName(linkRefPath.getName());
    TableName linkTableName = p.getFirst();
    String linkRegionName = p.getSecond();
    String hfileName = getBackReferenceFileName(linkRefPath.getParent());
    Path familyPath = linkRefPath.getParent().getParent();
    Path regionPath = familyPath.getParent();
    Path tablePath = regionPath.getParent();
    String linkName = createHFileLinkName(FSUtils.getTableName(tablePath),
        regionPath.getName(), hfileName);
    Path linkTableDir = FSUtils.getTableDir(rootDirlinkTableName);
    Path regionDir = HRegion.getRegionDir(linkTableDirlinkRegionName);
    return new Path(new Path(regionDirfamilyPath.getName()), linkName);
  }
    int separatorIndex = name.indexOf('.');
    String linkRegionName = name.substring(0, separatorIndex);
    String tableSubstr = name.substring(separatorIndex + 1)
        .replace('='.);
    TableName linkTableName = TableName.valueOf(tableSubstr);
    return new Pair<TableNameString>(linkTableNamelinkRegionName);
  }

  
Get the full path of the HFile referenced by the back reference

Parameters:
conf org.apache.hadoop.conf.Configuration to read for the archive directory name
linkRefPath Link Back Reference path
Returns:
full path of the referenced hfile
Throws:
java.io.IOException on unexpected error.
  public static Path getHFileFromBackReference(final Configuration conffinal Path linkRefPath)
      throws IOException {
    return getHFileFromBackReference(FSUtils.getRootDir(conf), linkRefPath);
  }
New to GrepCode? Check out our FAQ X