Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
Wrapper for input stream(s) that takes care of the interaction of FS and HBase checksums,
as well as closing streams. Initialization is not thread-safe, but normal operation is;
see method comments.
Two stream handles, one with and one without FS-level checksum.
HDFS checksum setting is on FS level, not single read level, so you have to keep two
FS objects and two handles open to interleave different reads freely, which is very sad.
This is what we do:
1) First, we need to read the trailer of HFile to determine checksum parameters.
We always use FS checksum to do that, so ctor opens
2.1) After that, if HBase checksum is not used, we'd just always use
2.2) If HBase checksum can be used, we'll open
. User MUST call prepareForBlockReader for that to happen;
if they don't, (2.1) will be the default.
3) The users can call
, and pass its result to
to get stream (if Java had out/pointer params we could
return both in one call). This stream is guaranteed to be set.
4) The first time HBase checksum fails, one would call
That will take lock, and open
. While this is going on, others will
continue to use the old stream; if they also want to fall back, they'll also call
, and block until
5) After some number of checksumOk() calls, we will go back to using HBase checksum.
We will have 2 handles; however we presume checksums fail so rarely that we don't care.
assert (path == null) != (link == null);
Prepares the streams for block reader. NOT THREAD SAFE. Must be called once, after any
reads finish and before any other reads start (what happens in reality is we read the
tail, then call this based on what's in the tail, then read blocks).
forceNoHBaseChecksum Force not using HBase checksum.
boolean useHBaseChecksum =
- Whether we are presently using HBase checksum.
Get the stream to use. Thread-safe.
useHBaseChecksum must be the value that shouldUseHBaseChecksum has returned
at some point in the past, otherwise the result is undefined.
Read from non-checksum stream failed, fall back to FS checksum. Thread-safe.
offCount For how many checksumOk calls to turn off the HBase checksum.
boolean partOfConvoy = false;
partOfConvoy = (this.stream != null);
Report that checksum was ok, so we may ponder going back to HBase checksum.
Close stream(s) if necessary.