Start line:  
End line:  

Snippet Preview

Snippet HTML Code

Stack Overflow Questions
Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
 
 package org.apache.hadoop.hbase.io;
 
 
 
Wrapper for input stream(s) that takes care of the interaction of FS and HBase checksums, as well as closing streams. Initialization is not thread-safe, but normal operation is; see method comments.
 
 public class FSDataInputStreamWrapper {
   private final HFileSystem hfs;
   private final Path path;
   private final FileLink link;
   private final boolean doCloseStreams;

  
Two stream handles, one with and one without FS-level checksum. HDFS checksum setting is on FS level, not single read level, so you have to keep two FS objects and two handles open to interleave different reads freely, which is very sad. This is what we do: 1) First, we need to read the trailer of HFile to determine checksum parameters. We always use FS checksum to do that, so ctor opens stream. 2.1) After that, if HBase checksum is not used, we'd just always use stream; 2.2) If HBase checksum can be used, we'll open streamNoFsChecksum, and close stream. User MUST call prepareForBlockReader for that to happen; if they don't, (2.1) will be the default. 3) The users can call shouldUseHBaseChecksum(), and pass its result to getStream(boolean) to get stream (if Java had out/pointer params we could return both in one call). This stream is guaranteed to be set. 4) The first time HBase checksum fails, one would call fallbackToFsChecksum(int). That will take lock, and open stream. While this is going on, others will continue to use the old stream; if they also want to fall back, they'll also call fallbackToFsChecksum(int), and block until stream is set. 5) After some number of checksumOk() calls, we will go back to using HBase checksum. We will have 2 handles; however we presume checksums fail so rarely that we don't care.
 
   private volatile FSDataInputStream stream = null;
   private volatile FSDataInputStream streamNoFsChecksum = null;
 
   // The configuration states that we should validate hbase checksums
   private boolean useHBaseChecksumConfigured;
 
   // Record the current state of this reader with respect to
   // validating checkums in HBase. This is originally set the same
   // value as useHBaseChecksumConfigured, but can change state as and when
   // we encounter checksum verification failures.
   private volatile boolean useHBaseChecksum;
 
   // In the case of a checksum failure, do these many succeeding
   // reads without hbase checksum verification.
   private volatile int hbaseChecksumOffCount = -1;
 
   public FSDataInputStreamWrapper(FileSystem fsPath paththrows IOException {
     this(fsnullpath);
   }
 
   public FSDataInputStreamWrapper(FileSystem fsFileLink linkthrows IOException {
     this(fslinknull);
   }
 
   private FSDataInputStreamWrapper(FileSystem fsFileLink linkPath paththrows IOException {
     assert (path == null) != (link == null);
     this. = path;
     this. = link;
     this. = true;
     // If the fs is not an instance of HFileSystem, then create an instance of HFileSystem
     // that wraps over the specified fs. In this case, we will not be able to avoid
     // checksumming inside the filesystem.
     this. = (fs instanceof HFileSystem) ? (HFileSystem)fs : new HFileSystem(fs);
 
     // Initially we are going to read the tail block. Open the reader w/FS checksum.
     this. = this. = false;
     this. = (link != null) ? link.open() : .open(path);
   }

  
Prepares the streams for block reader. NOT THREAD SAFE. Must be called once, after any reads finish and before any other reads start (what happens in reality is we read the tail, then call this based on what's in the tail, then read blocks).

Parameters:
forceNoHBaseChecksum Force not using HBase checksum.
  public void prepareForBlockReader(boolean forceNoHBaseChecksumthrows IOException {
    if ( == nullreturn;
    assert this. != null && !this.;
    boolean useHBaseChecksum =
        !forceNoHBaseChecksum && .useHBaseChecksum() && (.getNoChecksumFs() != );
    if (useHBaseChecksum) {
      FileSystem fsNc = .getNoChecksumFs();
      this. = ( != null) ? .open(fsNc) : fsNc.open();
      this. = this. = useHBaseChecksum;
      // Close the checksum stream; we will reopen it if we get an HBase checksum failure.
      this..close();
      this. = null;
    }
  }

  
For use in tests.
    this(fsdisfsdis);
  }

  
For use in tests.
     = false;
     = fsdis;
     = noChecksum;
     = null;
     = null;
     = null;
  }

  

Returns:
Whether we are presently using HBase checksum.
  public boolean shouldUseHBaseChecksum() {
    return this.;
  }

  
Get the stream to use. Thread-safe.

Parameters:
useHBaseChecksum must be the value that shouldUseHBaseChecksum has returned at some point in the past, otherwise the result is undefined.
  public FSDataInputStream getStream(boolean useHBaseChecksum) {
    return useHBaseChecksum ? this. : this.;
  }

  
Read from non-checksum stream failed, fall back to FS checksum. Thread-safe.

Parameters:
offCount For how many checksumOk calls to turn off the HBase checksum.
  public FSDataInputStream fallbackToFsChecksum(int offCountthrows IOException {
    // checksumOffCount is speculative, but let's try to reset it less.
    boolean partOfConvoy = false;
    if (this. == null) {
      synchronized () {
        partOfConvoy = (this. != null);
        if (!partOfConvoy) {
          this. = ( != null) ? .open() : .open();
        }
      }
    }
    if (!partOfConvoy) {
      this. = false;
      this. = offCount;
    }
    return this.;
  }

  
Report that checksum was ok, so we may ponder going back to HBase checksum.
  public void checksumOk() {
    if (this. && !this.
        && (this.-- < 0)) {
      // The stream we need is already open (because we were using HBase checksum in the past).
      assert this. != null;
      this. = true;
    }
  }

  
Close stream(s) if necessary.
  public void close() throws IOException {
    if (!return;
    try {
      if ( !=  &&  != null) {
        .close();
         = null;
      }
    } finally {
      if ( != null) {
        .close();
         = null;
      }
    }
  }
  public HFileSystem getHfs() {
    return this.;
  }
New to GrepCode? Check out our FAQ X