Big data hadoop practice test2

Thank you very much for taking test

Test your caliber

1. In a MapReduce job, you want each of your input files processed by a single map task. How do you configure a MapReduce job so that a single map task processes each input file regardless of how many blocks the input file occupies?
Increase the parameter that controls minimum split size in the job configuration.	Write a custom MapRunner that iterates over all key-value pairs in the entire file.
Set the number of mappers equal to the number of input files you want to process.	Write a custom FileInputFormat and override the method isSplitable to always return false.

2. Which process describes the lifecycle of a Mapper?
The JobTracker calls the TaskTracker’s configure () method, then its map () method and finally its close () method.	The TaskTracker spawns a new Mapper to process all records in a single input split.
The TaskTracker spawns a new Mapper to process each key-value pair.	The JobTracker spawns a new Mapper to process all records in a single file.

3. Determine which best describes when the reduce method is first called in a MapReduce job?
Reducers start copying intermediate key-value pairs from each Mapper as soon as it has completed. The programmer can configure in the job what percentage of the intermediate data should arrive before the reduce method begins.	Reducers start copying intermediate key-value pairs from each Mapper as soon as it has completed. The reduce method is called only after all intermediate data has been copied and sorted.
Reduce methods and map methods all start at the beginning of a job, in order to provide optimal performance for map-only or reduce-only jobs.	Reducers start copying intermediate key-value pairs from each Mapper as soon as it has completed. The reduce method is called as soon as the intermediate key-value pairs start to arrive.

4. You have written a Mapper which invokes the following five calls to the OutputColletor.collect method: output.collect (new Text (“Apple”), new Text (“Red”) ) ; output.collect (new Text (“Banana”), new Text (“Yellow”) ) ; output.collect (new Text (“Apple”), new Text (“Yellow”) ) ; output.collect (new Text (“Cherry”), new Text (“Red”) ) ; output.collect (new Text (“Apple”), new Text (“Green”) ) ; How many times will the Reducer’s reduce method be invoked?
6	3
1	0

5. To process input key-value pairs, your mapper needs to lead a 512 MB data file in memory. What is the best way to accomplish this?
Serialize the data file, insert in it the JobConf object, and read the data into memory in the configure method of the mapper.	Place the data file in the DistributedCache and read the data into memory in the map method of the mapper.
Place the data file in the DataCache and read the data into memory in the configure method of the mapper.	Place the data file in the DistributedCache and read the data into memory in the configure method of the mapper.

6. In a MapReduce job, the reducer receives all values associated with same key. Which statement best describes the ordering of these values?
The values are in sorted order.	The values are arbitrarily ordered, and the ordering may vary from run to run of the same MapReduce job.
The values are arbitrary ordered, but multiple runs of the same MapReduce job will always have the same ordering.	Since the values come from mapper outputs, the reducers will receive contiguous sections of sorted values.

7. You need to create a job that does frequency analysis on input data. You will do this by writing a Mapper that uses TextInputFormat and splits each value (a line of text from an input file) into individual characters. For each one of these characters, you will emit the character as a key and an InputWritable as the value. As this will produce proportionally more intermediate data than input data, which two resources should you expect to be bottlenecks?
Processor and network I/O	Disk I/O and network I/O
Processor and RAM	Processor and disk I/O

8. You want to count the number of occurrences for each unique word in the supplied input data. You’ve decided to implement this by having your mapper tokenize each word and emit a literal value 1, and then have your reducer increment a counter for each literal 1 it receives. After successful implementing this, it occurs to you that you could optimize this by specifying a combiner. Will you be able to reuse your existing Reduces as your combiner in this case and why or why not?
Yes, because the sum operation is both associative and commutative and the input and output types to the reduce method match.	No, because the sum operation in the reducer is incompatible with the operation of a Combiner.
No, because the Reducer and Combiner are separate interfaces.	No, because the Combiner is incompatible with a mapper which doesn’t use the same data type for both the key and value.

9. Your client application submits a MapReduce job to your Hadoop cluster. Identify the Hadoop daemon on which the Hadoop framework will look for an available slot schedule a MapReduce operation.
TaskTracker	NameNode
DataNode	JobTracker

10. Which project gives you a distributed, Scalable, data store that allows you random, realtime read/write access to hundreds of terabytes of data?
HBase	Hue
Pig	Hive

11. What is a SequenceFile?
A Sequence Filecontains a binary encoding of an arbitrary number of homo geneous writable objects.	A Sequence Filecontains a binary encoding of an arbitrary number of hetero geneous writable objects.
A Sequence Filecontains a binary encoding of an arbitrary number of Writable Comparable objects, in sorted order.	A Sequence Filecontains a binary encoding of an arbitrary number key-value pairs. Each key must be the same type. Each value must be same type.

12. In a MapReduce job, you want each of you input files processed by a single map task. How do you configure a MapReduce job so that a single map task processes each input file regardless of how many blocks the input file occupies?
Increase the parameter that controls minimum split size in the job configuration.	Write a custom MapRunner that iterates over all key-value pairs in the entire file.
Set the number of mappers equal to the number of input files you want to process.	Write a custom FileInputFormat and override the method isSplittable to always return false.

13. Which of the following best describes the workings of TextInputFormat?
Input file splits may cross line breaks. A line thatcrosses tile splits is ignored.	The input file is split exactly at the line breaks, so each Record Reader will read a series of complete lines.
Input file splits may cross line breaks. A line that crosses file splits is read by the RecordReaders of both splits containing the brokenline.	Input file splits may cross line breaks. A line that crosses file splits is read by the RecordReader of the split that contains the end of the broken line.

14. Which of the following statements most accurately describes the relationship between MapReduce and Pig?
Pig provides additional capabilities that allow certain types of data manipulation not possible with MapReduce.	Pig provides no additional capabilities to MapReduce. Pig programs are executed as MapReduce jobs via the Pig interpreter.
Pig programs rely on MapReduce but are extensible, allowing developers to do specialpurpose processing not provided by MapReduce.	Pig provides the additional capability of allowing you to control the flow of multiple MapReduce jobs.

15. Will settings using Java API overwrite values in configuration files?
No. The configuration settings in the configuration file takes precedence	Yes. The configuration settings using Java API take precedence
It depends when the developer reads the configuration file. If it is read first then no.	Only global configuration settings are captured in configuration files on namenode. There are only a very few job parameters that can be set using Java API.

16. What is AVRO?
Avro is a java serialization library	Avro is a java compression library
Avro is a java library that create splittable files	None of these answers are correct

17. Can you run Map - Reduce jobs directly on Avro data?
Yes, Avro was specifically designed for data processing via Map-Reduce	Yes, but additional extensive coding is required
No, Avro was specifically designed for data storage only	Avro specifies metadata that allows easier data access. This data cannot be used as part of mapreduce execution, rather input specification only.

18. What is distributed cache?
The distributed cache is special component on namenode that will cache frequently used data for faster client response. It is used during reduce step.	The distributed cache is special component on datanode that will cache frequently used data for faster client response. It is used during map step.
The distributed cache is a component that caches java objects.	The distributed cache is a component that allows developers to deploy jars for Map-Reduce processing.

19. What is the best performance one can expect from a Hadoop cluster?
The best performance expectation one can have is measured in seconds. This is because Hadoop can only be used for batch processing	The best performance expectation one can have is measured in milliseconds. This is because Hadoop executes in parallel across so many machines
The best performance expectation one can have is measured in minutes. This is because Hadoop can only be used for batch processing	It depends on on the design of the map-reduce program, how many machines in the cluster, and the amount of data being retrieved

20. What is writable?
Writable is a java interface that needs to be implemented for streaming data to remote servers.	Writable is a java interface that needs to be implemented for HDFS writes.
Writable is a java interface that needs to be implemented for MapReduce processing.	None of these answers are correct.

What is reduce - side join?

A. Reduce-side join is a technique to eliminate data from initial data set at reduce step
B. Reduce-side join is a technique for merging data from different sources based on a specific key.
C. Reduce-side join is a set of API to merge data from different sources.
D. None of these answers are correct

Answer: B

What is map - side join?

A . Map-side join is done in the map phase and done in memory
B . Map-side join is a technique in which data is eliminated at the map step
C . Map-side join is a form of map-reduce API which joins data from different locations
D . None of these answers are correct

Answer: A

How can you use binary data in MapReduce?

A. Binary data can be used directly by a map-reduce job. Often binary data is added to a sequence file.
B. Binary data cannot be used by Hadoop fremework. Binary data should be converted to a Hadoop compatible format prior to loading.
C. Binary can be used in map-reduce only with very limited functionlity. It cannot be used as a key for example.
D. Hadoop can freely use binary files with map-reduce jobs so long as the files have headers

Answer: A

What are map files and why are they important?

A. Map files are stored on the namenode and capture the metadata for all blocks on a particular rack.
This is how Hadoop is "rack aware"
B. Map files are the files that show how the data is distributed in the Hadoop cluster.
C. Map files are generated by Map-Reduce after the reduce step. They show the task distribution during job execution
D. Map files are sorted sequence files that also have an index. The index allows fast data look up.

Answer: D

What are sequence files and why are they important?

A. Sequence files are binary format files that are compressed and are splitable. They are often used in high-performance map-reduce jobs
B. Sequence files are a type of the file in the Hadoop framework that allow data to be sorted
C. Sequence files are intermediate files that are created by Hadoop after the map step
D. Both B and C are correct

Answer: A

What are supported programming languages for Map Reduce?

A. The most common programming language is Java, but scripting languages are also supported via Hadoop streaming.
B. Any programming language that can comply with Map Reduce concept can be supported.
C. Only Java supported since Hadoop was written in Java.
D. Currently Map Reduce supports Java, C, C++ and COBOL.

Answer: A

How many states does Writable interface defines

A. Two
B. Four
C. Three
D. None of the above

Answer: A

Which method of the FileSystem object is used for reading a file in HDFS

A. open()
B. access()
C. select()
D. None of the above

Answer: A

RPC means

A. Remote processing call
B. Remote process call
C. Remote procedure call
D. None of the above

Answer: C

The switch given to “hadoop fs” command for detailed help is

A. -show
B. -help
C. -?
D. None of the above

Answer: B

The size of block in HDFS is

A. 512 bytes
B. 64 MB
C. 1024 KB
D. None of the above

Answer: B

Which MapReduce phase is theoretically able to utilize features of the underlying file system in order to optimize parallel execution?

A. Split
B. Map
C. Combine

Ans: A

What is the input to the Reduce function?

A. One key and a list of all values associated with that key.
B. One key and a list of some values associated with that key.
C. An arbitrarily sized list of key/value pairs.

Ans: A

How can a distributed filesystem such as HDFS provide opportunities for optimization of a MapReduce operation?

A. Data represented in a distributed filesystem is already sorted.
B. Distributed filesystems must always be resident in memory, which is much faster than disk.
C. Data storage and processing can be co-located on the same node, so that most input data relevant to Map or Reduce will be present on local disks or cache.
D. A distributed filesystem makes random access faster because of the presence of a dedicated node serving file metadata.

Ans: D

Which of the following MapReduce execution frameworks focus on execution in sharedmemory environments?

A. Hadoop
B. Twister
C. Phoenix

Ans: C

What is the implementation language of the Hadoop MapReduce framework?

A. Java
B. C
C. FORTRAN
D. Python

Ans: A

The Combine stage, if present, must perform the same aggregation operation as Reduce.

A. True
B. False

Ans: B

Which of the following statements most accurately describes the general approach to error recovery when using MapReduce?

A. Ranger
B. Longhorn
C. Lonestar
D. Spur

Ans: A

Which MapReduce stage serves as a barrier, where all previous stages must be completed before it may proceed?

A. Combine
B. Group (a.k.a. 'shuffle')
C. Reduce
D. Write

Ans: A