最新の[2022年02月22日]Databricks Associate-Developer-Apache-Spark試験練習テスト最高成績で最速合格をゲットせよ！ [Q64-Q80]

最新の[2022年02月22日]Databricks Associate-Developer-Apache-Spark試験練習テスト最高成績で最速合格をゲットせよ！

これを使えば必ず合格させる問題集でDatabricks Associate-Developer-Apache-Spark

質問 64
Which of the following describes how Spark achieves fault tolerance?

A. Spark is only fault-tolerant if this feature is specifically enabled via the spark.fault_recovery.enabled property.
B. If an executor on a worker node fails while calculating an RDD, that RDD can be recomputed by another executor using the lineage.
C. Spark builds a fault-tolerant layer on top of the legacy RDD data system, which by itself is not fault tolerant.
D. Due to the mutability of DataFrames after transformations, Spark reproduces them using observed lineage in case of worker node failure.
E. Spark helps fast recovery of data in case of a worker fault by providing the MEMORY_AND_DISK storage level option.

正解: B

解説:
Explanation
Due to the mutability of DataFrames after transformations, Spark reproduces them using observed lineage in case of worker node failure.
Wrong - Between transformations, DataFrames are immutable. Given that Spark also records the lineage, Spark can reproduce any DataFrame in case of failure. These two aspects are the key to understanding fault tolerance in Spark.
Spark builds a fault-tolerant layer on top of the legacy RDD data system, which by itself is not fault tolerant.
Wrong. RDD stands for Resilient Distributed Dataset and it is at the core of Spark and not a "legacy system".
It is fault-tolerant by design.
Spark helps fast recovery of data in case of a worker fault by providing the MEMORY_AND_DISK storage level option.
This is not true. For supporting recovery in case of worker failures, Spark provides "_2", "_3", and so on, storage level options, for example MEMORY_AND_DISK_2. These storage levels are specifically designed to keep duplicates of the data on multiple nodes. This saves time in case of a worker fault, since a copy of the data can be used immediately, vs. having to recompute it first.
Spark is only fault-tolerant if this feature is specifically enabled via the spark.fault_recovery.enabled property.
No, Spark is fault-tolerant by design.

質問 65
Which is the highest level in Spark's execution hierarchy?

A. Slot
B. Stage
C. Executor
D. Job
E. Task

正解: D

質問 66
Which of the following describes the conversion of a computational query into an execution plan in Spark?

A. The executed physical plan depends on a cost optimization from a previous stage.
B. Spark uses the catalog to resolve the optimized logical plan.
C. The catalog assigns specific resources to the optimized memory plan.
D. The catalog assigns specific resources to the physical plan.
E. Depending on whether DataFrame API or SQL API are used, the physical plan may differ.

正解: A

解説:
Explanation
The executed physical plan depends on a cost optimization from a previous stage.
Correct! Spark considers multiple physical plans on which it performs a cost analysis and selects the final physical plan in accordance with the lowest-cost outcome of that analysis. That final physical plan is then executed by Spark.
Spark uses the catalog to resolve the optimized logical plan.
No. Spark uses the catalog to resolve the unresolved logical plan, but not the optimized logical plan. Once the unresolved logical plan is resolved, it is then optimized using the Catalyst Optimizer.
The optimized logical plan is the input for physical planning.
The catalog assigns specific resources to the physical plan.
No. The catalog stores metadata, such as a list of names of columns, data types, functions, and databases.
Spark consults the catalog for resolving the references in a logical plan at the beginning of the conversion of the query into an execution plan. The result is then an optimized logical plan.
Depending on whether DataFrame API or SQL API are used, the physical plan may differ.
Wrong - the physical plan is independent of which API was used. And this is one of the great strengths of Spark!
The catalog assigns specific resources to the optimized memory plan.
There is no specific "memory plan" on the journey of a Spark computation.
More info: Spark's Logical and Physical plans ... When, Why, How and Beyond. | by Laurent Leturgez | datalex | Medium

質問 67
The code block displayed below contains an error. The code block is intended to perform an outer join of DataFrames transactionsDf and itemsDf on columns productId and itemId, respectively.
Find the error.
Code block:
transactionsDf.join(itemsDf, [itemsDf.itemId, transactionsDf.productId], "outer")

A. The term [itemsDf.itemId, transactionsDf.productId] should be replaced by itemsDf.itemId == transactionsDf.productId.
B. The "outer" argument should be eliminated from the call and join should be replaced by joinOuter.
C. The "outer" argument should be eliminated, since "outer" is the default join type.
D. The join type needs to be appended to the join() operator, like join().outer() instead of listing it as the last argument inside the join() call.
E. The term [itemsDf.itemId, transactionsDf.productId] should be replaced by itemsDf.col("itemId") == transactionsDf.col("productId").

正解: A

解説:
Explanation
Correct code block:
transactionsDf.join(itemsDf, itemsDf.itemId == transactionsDf.productId, "outer") Static notebook | Dynamic notebook: See test 1 (https://flrs.github.io/spark_practice_tests_code/#1/33.html ,
https://bit.ly/sparkpracticeexams_import_instructions)

質問 68
The code block shown below should return an exact copy of DataFrame transactionsDf that does not include rows in which values in column storeId have the value 25. Choose the answer that correctly fills the blanks in the code block to accomplish this.

A. transactionsDf.drop(transactionsDf.storeId==25)
B. transactionsDf.filter(transactionsDf.storeId==25)
C. transactionsDf.where(transactionsDf.storeId!=25)
D. transactionsDf.remove(transactionsDf.storeId==25)
E. transactionsDf.select(transactionsDf.storeId!=25)

正解: C

解説:
Explanation
transactionsDf.where(transactionsDf.storeId!=25)
Correct. DataFrame.where() is an alias for the DataFrame.filter() method. Using this method, it is straightforward to filter out rows that do not have value 25 in column storeId.
transactionsDf.select(transactionsDf.storeId!=25)
Wrong. The select operator allows you to build DataFrames column-wise, but when using it as shown, it does not filter out rows.
transactionsDf.filter(transactionsDf.storeId==25)
Incorrect. Although the filter expression works for filtering rows, the == in the filtering condition is inappropriate. It should be != instead.
transactionsDf.drop(transactionsDf.storeId==25)
No. DataFrame.drop() is used to remove specific columns, but not rows, from the DataFrame.
transactionsDf.remove(transactionsDf.storeId==25)
False. There is no DataFrame.remove() operator in PySpark.
More info: pyspark.sql.DataFrame.where - PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 3

質問 69
In which order should the code blocks shown below be run in order to assign articlesDf a DataFrame that lists all items in column attributes ordered by the number of times these items occur, from most to least often?
Sample of DataFrame articlesDf:
1.+------+-----------------------------+-------------------+
2.|itemId|attributes |supplier |
3.+------+-----------------------------+-------------------+
4.|1 |[blue, winter, cozy] |Sports Company Inc.|
5.|2 |[red, summer, fresh, cooling]|YetiX |
6.|3 |[green, summer, travel] |Sports Company Inc.|
7.+------+-----------------------------+-------------------+

A. 5, 2
B. 2, 3, 4
C. 4, 5
D. 2, 5, 4
E. 1. articlesDf = articlesDf.groupby("col")
2. articlesDf = articlesDf.select(explode(col("attributes")))
3. articlesDf = articlesDf.orderBy("count").select("col")
4. articlesDf = articlesDf.sort("count",ascending=False).select("col")
5. articlesDf = articlesDf.groupby("col").count()
F. 2, 5, 3

正解: B

質問 70
Which of the following statements about lazy evaluation is incorrect?

A. Lineages allow Spark to coalesce transformations into stages
B. Execution is triggered by transformations.
C. Spark will fail a job only during execution, but not during definition.
D. Predicate pushdown is a feature resulting from lazy evaluation.
E. Accumulators do not change the lazy evaluation model of Spark.

正解: B

解説:
Explanation
Execution is triggered by transformations.
Correct. Execution is triggered by actions only, not by transformations.
Lineages allow Spark to coalesce transformations into stages.
Incorrect. In Spark, lineage means a recording of transformations. This lineage enables lazy evaluation in Spark.
Predicate pushdown is a feature resulting from lazy evaluation.
Wrong. Predicate pushdown means that, for example, Spark will execute filters as early in the process as possible so that it deals with the least possible amount of data in subsequent transformations, resulting in a performance improvements.
Accumulators do not change the lazy evaluation model of Spark.
Incorrect. In Spark, accumulators are only updated when the query that refers to the is actually executed. In other words, they are not updated if the query is not (yet) executed due to lazy evaluation.
Spark will fail a job only during execution, but not during definition.
Wrong. During definition, due to lazy evaluation, the job is not executed and thus certain errors, for example reading from a non-existing file, cannot be caught. To be caught, the job needs to be executed, for example through an action.

質問 71
The code block shown below should return a new 2-column DataFrame that shows one attribute from column attributes per row next to the associated itemName, for all suppliers in column supplier whose name includes Sports. Choose the answer that correctly fills the blanks in the code block to accomplish this.
Sample of DataFrame itemsDf:
1.+------+----------------------------------+-----------------------------+-------------------+
2.|itemId|itemName |attributes |supplier |
3.+------+----------------------------------+-----------------------------+-------------------+
4.|1 |Thick Coat for Walking in the Snow|[blue, winter, cozy] |Sports Company Inc.|
5.|2 |Elegant Outdoors Summer Dress |[red, summer, fresh, cooling]|YetiX |
6.|3 |Outdoors Backpack |[green, summer, travel] |Sports Company Inc.|
7.+------+----------------------------------+-----------------------------+-------------------+ Code block:
itemsDf.__1__(__2__).select(__3__, __4__)

A. 1. filter
2. col("supplier").contains("Sports")
3. "itemName"
4. explode("attributes")
B. 1. where
2. "Sports".isin(col("Supplier"))
3. "itemName"
4. array_explode("attributes")
C. 1. where
2. col("supplier").contains("Sports")
3. "itemName"
4. "attributes"
D. 1. filter
2. col("supplier").isin("Sports")
3. "itemName"
4. explode(col("attributes"))
E. 1. where
2. col(supplier).contains("Sports")
3. explode(attributes)
4. itemName

正解: A

解説:
Explanation
Output of correct code block:
+----------------------------------+------+
|itemName |col |
+----------------------------------+------+
|Thick Coat for Walking in the Snow|blue |
|Thick Coat for Walking in the Snow|winter|
|Thick Coat for Walking in the Snow|cozy |
|Outdoors Backpack |green |
|Outdoors Backpack |summer|
|Outdoors Backpack |travel|
+----------------------------------+------+
The key to solving this question is knowing about Spark's explode operator. Using this operator, you can extract values from arrays into single rows. The following guidance steps through the answers systematically from the first to the last gap. Note that there are many ways to solving the gap questions and filtering out wrong answers, you do not always have to start filtering out from the first gap, but can also exclude some answers based on obvious problems you see with them.
The answers to the first gap present you with two options: filter and where. These two are actually synonyms in PySpark, so using either of those is fine. The answer options to this gap therefore do not help us in selecting the right answer.
The second gap is more interesting. One answer option includes "Sports".isin(col("Supplier")). This construct does not work, since Python's string does not have an isin method. Another option contains col(supplier). Here, Python will try to interpret supplier as a variable. We have not set this variable, so this is not a viable answer. Then, you are left with answers options that include col ("supplier").contains("Sports") and col("supplier").isin("Sports"). The question states that we are looking for suppliers whose name includes Sports, so we have to go for the contains operator here.
We would use the isin operator if we wanted to filter out for supplier names that match any entries in a list of supplier names.
Finally, we are left with two answers that fill the third gap both with "itemName" and the fourth gap either with explode("attributes") or "attributes". While both are correct Spark syntax, only explode ("attributes") will help us achieve our goal. Specifically, the question asks for one attribute from column attributes per row - this is what the explode() operator does.
One answer option also includes array_explode() which is not a valid operator in PySpark.
More info: pyspark.sql.functions.explode - PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 3

質問 72
The code block displayed below contains an error. The code block is intended to write DataFrame transactionsDf to disk as a parquet file in location /FileStore/transactions_split, using column storeId as key for partitioning. Find the error.
Code block:
transactionsDf.write.format("parquet").partitionOn("storeId").save("/FileStore/transactions_split")A.

A. Partitioning data by storeId is possible with the bucketBy expression, so partitionOn should be replaced by bucketBy.
B. Partitioning data by storeId is possible with the partitionBy expression, so partitionOn should be replaced by partitionBy.
C. The format("parquet") expression is inappropriate to use here, "parquet" should be passed as first argument to the save() operator and "/FileStore/transactions_split" as the second argument.
D. The format("parquet") expression should be removed and instead, the information should be added to the write expression like so: write("parquet").
E. partitionOn("storeId") should be called before the write operation.

正解: B

解説:
Explanation
Correct code block:
transactionsDf.write.format("parquet").partitionBy("storeId").save("/FileStore/transactions_split") More info: partition by - Reading files which are written using PartitionBy or BucketBy in Spark - Stack Overflow Static notebook | Dynamic notebook: See test 1

質問 73
The code block displayed below contains an error. The code block should produce a DataFrame with color as the only column and three rows with color values of red, blue, and green, respectively.
Find the error.
Code block:
1.spark.createDataFrame([("red",), ("blue",), ("green",)], "color")
Instead of calling spark.createDataFrame, just DataFrame should be called.

A. The commas in the tuples with the colors should be eliminated.
B. The colors red, blue, and green should be expressed as a simple Python list, and not a list of tuples.
C. The "color" expression needs to be wrapped in brackets, so it reads ["color"].
D. Instead of color, a data type should be specified.

正解: C

解説:
Explanation
Correct code block:
spark.createDataFrame([("red",), ("blue",), ("green",)], ["color"])
The createDataFrame syntax is not exactly straightforward, but luckily the documentation (linked below) provides several examples on how to use it. It also shows an example very similar to the code block presented here which should help you answer this question correctly.
More info: pyspark.sql.SparkSession.createDataFrame - PySpark 3.1.2 documentation Static notebook | Dynamic notebook: See test 2

質問 74
The code block displayed below contains an error. The code block is intended to return all columns of DataFrame transactionsDf except for columns predError, productId, and value. Find the error.
Excerpt of DataFrame transactionsDf:
transactionsDf.select(~col("predError"), ~col("productId"), ~col("value"))

A. The column names in the select operator should not be strings and wrapped in the col operator, so they should be expressed like select(~col(predError), ~col(productId), ~col(value)).
B. The select operator should be replaced by the drop operator and the arguments to the drop operator should be column names predError, productId and value as strings.
(Correct)
C. The select operator should be replaced by the drop operator.
D. The select operator should be replaced by the drop operator and the arguments to the drop operator should be column names predError, productId and value wrapped in the col operator so they should be expressed like drop(col(predError), col(productId), col(value)).
E. The select operator should be replaced with the deselect operator.

正解: B

解説:
Explanation
Correct code block:
transactionsDf.drop("predError", "productId", "value")
Static notebook | Dynamic notebook: See test 1

質問 75
The code block displayed below contains an error. The code block should save DataFrame transactionsDf at path path as a parquet file, appending to any existing parquet file. Find the error.
Code block:

A. transactionsDf.format("parquet").option("mode", "append").save(path)
B. The code block is missing a reference to the DataFrameWriter.
C. The mode option should be omitted so that the command uses the default mode.
D. save() is evaluated lazily and needs to be followed by an action.
E. Given that the DataFrame should be saved as parquet file, path is being passed to the wrong method.
F. The code block is missing a bucketBy command that takes care of partitions.

正解: B

解説:
Explanation
Correct code block:
transactionsDf.write.format("parquet").option("mode", "append").save(path)

A. itemsDf.filter(!col('supplier').contains('X')).select(col('supplier')).unique()
B. itemsDf.filter(~col('supplier').contains('X')).select('supplier').distinct()
C. itemsDf.filter(col(supplier).not_contains('X')).select(supplier).distinct()
D. itemsDf.select(~col('supplier').contains('X')).distinct()
E. itemsDf.filter(not(col('supplier').contains('X'))).select('supplier').unique()

正解: B

解説:
Explanation
Output of correct code block:
+-------------------+
| supplier|
+-------------------+
|Sports Company Inc.|
+-------------------+
Key to managing this question is understand which operator to use to do the opposite of an operation
- the ~ (not) operator. In addition, you should know that there is no unique() method.
Static notebook | Dynamic notebook: See test 1

質問 77
The code block displayed below contains an error. The code block should return the average of rows in column value grouped by unique storeId. Find the error.
Code block:
transactionsDf.agg("storeId").avg("value")

A. All column names should be wrapped in col() operators.
B. agg should be replaced by groupBy.
C. Instead of avg("value"), avg(col("value")) should be used.
D. "storeId" and "value" should be swapped.
E. The avg("value") should be specified as a second argument to agg() instead of being appended to it.

正解: B

解説:
Explanation
Static notebook | Dynamic notebook: See test 1
(https://flrs.github.io/spark_practice_tests_code/#1/30.html ,
https://bit.ly/sparkpracticeexams_import_instructions)

質問 78
The code block displayed below contains an error. The code block should combine data from DataFrames itemsDf and transactionsDf, showing all rows of DataFrame itemsDf that have a matching value in column itemId with a value in column transactionsId of DataFrame transactionsDf. Find the error.
Code block:
itemsDf.join(itemsDf.itemId==transactionsDf.transactionId)

A. The join method is inappropriate.
B. The merge method should be used instead of join.
C. The join statement is incomplete.
D. The union method should be used instead of join.
E. The join expression is malformed.

正解: C

解説:
Explanation
Correct code block:
itemsDf.join(transactionsDf, itemsDf.itemId==transactionsDf.transactionId) The join statement is incomplete.
Correct! If you look at the documentation of DataFrame.join() (linked below), you see that the very first argument of join should be the DataFrame that should be joined with. This first argument is missing in the code block.
The join method is inappropriate.
No. By default, DataFrame.join() uses an inner join. This method is appropriate for the scenario described in the question.
The join expression is malformed.
Incorrect. The join expression itemsDf.itemId==transactionsDf.transactionId is correct syntax.
The merge method should be used instead of join.
False. There is no DataFrame.merge() method in PySpark.
The union method should be used instead of join.
Wrong. DataFrame.union() merges rows, but not columns as requested in the question.
More info: pyspark.sql.DataFrame.join - PySpark 3.1.2 documentation, pyspark.sql.DataFrame.union - PySpark 3.1.2 documentation Static notebook | Dynamic notebook: See test 3

質問 79
The code block displayed below contains an error. The code block should write DataFrame transactionsDf as a parquet file to location filePath after partitioning it on column storeId. Find the error.
Code block:
transactionsDf.write.partitionOn("storeId").parquet(filePath)

A. The operator should use the mode() option to configure the DataFrameWriter so that it replaces any existing files at location filePath.
B. The partitioning column as well as the file path should be passed to the write() method of DataFrame transactionsDf directly and not as appended commands as in the code block.
C. The partitionOn method should be called before the write method.
D. Column storeId should be wrapped in a col() operator.
E. No method partitionOn() exists for the DataFrame class, partitionBy() should be used instead.

正解: E

解説:
Explanation
No method partitionOn() exists for the DataFrame class, partitionBy() should be used instead.
Correct! Find out more about partitionBy() in the documentation (linked below).
The operator should use the mode() option to configure the DataFrameWriter so that it replaces any existing files at location filePath.
No. There is no information about whether files should be overwritten in the question.
The partitioning column as well as the file path should be passed to the write() method of DataFrame transactionsDf directly and not as appended commands as in the code block.
Incorrect. To write a DataFrame to disk, you need to work with a DataFrameWriter object which you get access to through the DataFrame.writer property - no parentheses involved.
Column storeId should be wrapped in a col() operator.
No, this is not necessary - the problem is in the partitionOn command (see above).
The partitionOn method should be called before the write method.
Wrong. First of all partitionOn is not a valid method of DataFrame. However, even assuming partitionOn would be replaced by partitionBy (which is a valid method), this method is a method of DataFrameWriter and not of DataFrame. So, you would always have to first call DataFrame.write to get access to the DataFrameWriter object and afterwards call partitionBy.
More info: pyspark.sql.DataFrameWriter.partitionBy - PySpark 3.1.2 documentation Static notebook | Dynamic notebook: See test 3

質問 80
......

正真正銘のベスト問題集資料を使おうAssociate-Developer-Apache-Sparkオンライン練習試験：https://jp.fast2test.com/Associate-Developer-Apache-Spark-premium-file.html

最大一年間毎日更新されるAssociate-Developer-Apache-Sparkブレーン問題集：https://drive.google.com/open?id=1iXTRIAawpQAStgIN7QLWpHI0IyLvHG7q

関するブログ

もっと

Databricks認証 Associate-Developer-Apache-Spark試験問題集

弊社を連絡する

我々は１２時間以内ですべてのお問い合わせを答えます。

我々の働いている時間： ( GMT 0:00-15:00 )
月曜日から土曜日まで

サポート： 現在連絡

English Deutsch 繁体中文 한국어

最新の[2022年02月22日]Databricks Associate-Developer-Apache-Spark試験練習テスト最高成績で最速合格をゲットせよ！ [Q64-Q80]

関するブログ

弊社を連絡する

フレンドリーリンク

トップ試験