無料セールまもなく終了！100%有効Databricks-Certified-Data-Engineer-Associate試験問題集に102問題と解答が待ってます [Q12-Q37]

無料セールまもなく終了！100%有効Databricks-Certified-Data-Engineer-Associate試験問題集に102問題と解答が待ってます

検証済みDatabricks-Certified-Data-Engineer-Associate問題集と解答であなたを合格確定させるDatabricks Certification試験解答！

この試験は、60の複数選択の質問で構成されており、120分間の期間があります。試験で説明されているトピックには、Databricksアーキテクチャ、データ摂取、データ処理、データ変換、データモデリング、データウェアハウジングが含まれます。この試験は、データエンジニアリング、データエンジニアリングのタスクのためにDataBricksの操作の経験があるデータエンジニア、データアーキテクト、データアナリスト、およびデータサイエンティストを対象としています。この認定を取得することにより、候補者はDatabricksの専門知識を示し、データエンジニアリングの分野でのキャリアの見通しを強化できます。

質問 # 12
A data engineer is attempting to drop a Spark SQL table my_table. The data engineer wants to delete all table metadata and data.
They run the following command:
DROP TABLE IF EXISTS my_table
While the object no longer appears when they run SHOW TABLES, the data files still exist.
Which of the following describes why the data files still exist and the metadata files were deleted?

A. The table's data was smaller than 10 GB
B. The table did not have a location
C. The table was managed
D. The table was external
E. The table's data was larger than 10 GB

正解：D

質問 # 13
Which of the following tools is used by Auto Loader process data incrementally?

A. Data Explorer
B. Unity Catalog
C. Spark Structured Streaming
D. Checkpointing
E. Databricks SQL

正解：C

解説：
Auto Loader provides a Structured Streaming source called cloudFiles that can process new data files as they arrive in cloud storage without any additional setup. Auto Loader uses a scalable key-value store to track ingestion progress and ensure exactly-once semantics. Auto Loader can ingest various file formats and load them into Delta Lake tables. Auto Loader is recommended for incremental data ingestion with Delta Live Tables, which extends the functionality of Structured Streaming and allows you to write declarative Python or SQL code to deploy a production-quality data pipeline. Reference: What is Auto Loader?, What is Auto Loader? | Databricks on AWS, Solved: How does Auto Loader ingest data? - Databricks - 5629

質問 # 14
A data engineer needs to create a table in Databricks using data from their organization's existing SQLite database. They run the following command:
CREATE TABLE jdbc_customer360
USING
OPTIONS (
url "jdbc:sqlite:/customers.db", dbtable "customer360"
)
Which line of code fills in the above blank to successfully complete the task?

A. org.apache.spark.sql.sqlite
B. sqlite
C. autoloader
D. org.apache.spark.sql.jdbc

正解：D

解説：
To create a table in Databricks using data from an SQLite database, the correct syntax involves specifying the format of the data source. The format in the case of using JDBC (Java Database Connectivity) with SQLite is specified by the org.apache.spark.sql.jdbc format. This format allows Spark to interface with various relational databases through JDBC. Here is how the command should be structured:
CREATE TABLE jdbc_customer360
USING org.apache.spark.sql.jdbc
OPTIONS (
url "jdbc:sqlite:/customers.db",
dbtable "customer360"
)
The USING org.apache.spark.sql.jdbc line specifies that the JDBC data source is being used, enabling Spark to interact with the SQLite database via JDBC.
Reference:
Databricks documentation on JDBC: Connecting to SQL Databases using JDBC

質問 # 15
A data engineer only wants to execute the final block of a Python program if the Python variable day_of_week is equal to 1 and the Python variable review_period is True.
Which of the following control flow statements should the data engineer use to begin this conditionally executed code block?

A. if day_of_week = 1 and review_period:
B. if day_of_week == 1 and review_period == "True":
C. if day_of_week == 1 and review_period:
D. if day_of_week = 1 and review_period = "True":
E. if day_of_week = 1 & review_period: = "True":

正解：B

質問 # 16
A data engineer has left the organization. The data team needs to transfer ownership of the data engineer's Delta tables to a new data engineer. The new data engineer is the lead engineer on the data team.
Assuming the original data engineer no longer has access, which of the following individuals must be the one to transfer ownership of the Delta tables in Data Explorer?

A. New lead data engineer
B. Original data engineer
C. Workspace administrator
D. This transfer is not possible
E. Databricks account representative

正解：C

解説：
Explanation
https://docs.databricks.com/sql/admin/transfer-ownership.html

質問 # 17
A data engineer has a Job with multiple tasks that runs nightly. Each of the tasks runs slowly because the clusters take a long time to start.
Which of the following actions can the data engineer perform to improve the start up time for the clusters used for the Job?

A. They can use endpoints available in Databricks SQL
B. They can use clusters that are from a cluster pool
C. They can use jobs clusters instead of all-purpose clusters
D. They can configure the clusters to be single-node
E. They can configure the clusters to autoscale for larger data sizes

正解：B

解説：
The best action that the data engineer can perform to improve the start up time for the clusters used for the Job is to use clusters that are from a cluster pool. A cluster pool is a set of idle clusters that can be used by jobs or interactive sessions. By using a cluster pool, the data engineer can avoid the cluster creation time and reduce the latency of the tasks. Cluster pools also offer cost savings and resource efficiency, as they can be shared by multiple users and jobs.
Option A is not relevant, as endpoints available in Databricks SQL are used for creating and managing SQL analytics workloads, not for improving cluster start up time.
Option B is not correct, as jobs clusters and all-purpose clusters have similar start up times. Jobs clusters are clusters that are dedicated to run a single job and are terminated when the job is completed. All-purpose clusters are clusters that can be used for multiple purposes, such as interactive sessions, notebooks, or multiple jobs. Both types of clusters can benefit from using a cluster pool.
Option C is not advisable, as configuring the clusters to be single-node will reduce the parallelism and performance of the tasks. Single-node clusters are clusters that have only one worker node and are typically used for testing or development purposes. They are not suitable for running production jobs that require high scalability and fault tolerance.
Option E is not helpful, as configuring the clusters to autoscale for larger data sizes will not affect the start up time of the clusters. Autoscaling is a feature that allows clusters to dynamically adjust the number of worker nodes based on the workload. It can help optimize the resource utilization and cost efficiency of the clusters, but it does not speed up the cluster creation process.
References:
* Cluster Pools
* Jobs
* Clusters
* [Databricks Data Engineer Professional Exam Guide]

質問 # 18
A single Job runs two notebooks as two separate tasks. A data engineer has noticed that one of the notebooks is running slowly in the Job's current run. The data engineer asks a tech lead for help in identifying why this might be the case.
Which of the following approaches can the tech lead use to identify why the notebook is running slowly as part of the Job?

A. They can navigate to the Runs tab in the Jobs UI and click on the active run to review the processing notebook.
B. There is no way to determine why a Job task is running slowly.
C. They can navigate to the Tasks tab in the Jobs UI to immediately review the processing notebook.
D. They can navigate to the Runs tab in the Jobs UI to immediately review the processing notebook.
E. They can navigate to the Tasks tab in the Jobs UI and click on the active run to review the processing notebook.

正解：E

解説：
The Tasks tab in the Jobs UI shows the list of tasks that are part of a job, and allows the user to view the details of each task, such as the notebook path, the cluster configuration, the run status, and the duration. By clicking on the active run of a task, the user can access the Spark UI, the notebook output, and the logs of the task. These can help the user to identify the performance bottlenecks and errors in the task. The Runs tab in the Jobs UI only shows the summary of the job runs, such as the start time, the end time, the trigger, and the status. It does not provide the details of the individual tasks within a job run. Reference: Jobs UI, Monitor running jobs with a Job Run dashboard, How to optimize jobs performance

質問 # 19
Which of the following benefits of using the Databricks Lakehouse Platform is provided by Delta Lake?

A. The ability to set up alerts for query failures
B. The ability to manipulate the same data using a variety of languages
C. The ability to support batch and streaming workloads
D. The ability to distribute complex data operations
E. The ability to collaborate in real time on a single notebook

正解：C

解説：
Delta Lake is the optimized storage layer that provides the foundation for storing data and tables in the Databricks lakehouse. Delta Lake is fully compatible with Apache Spark APIs, and was developed for tight integration with Structured Streaming, allowing you to easily use a single copy of data for both batch and streaming operations and providing incremental processing at scale1. Delta Lake supports upserts using the merge operation, which enables you to efficiently update existing data or insert new data into your Delta tables2. Delta Lake also provides time travel capabilities, which allow you to query previous versions of your data or roll back to a specific point in time3. Reference: 1: What is Delta Lake? | Databricks on AWS 2: Upsert into a table using merge | Databricks on AWS 3: [Query an older snapshot of a table (time travel) | Databricks on AWS] Learn more
1learn.microsoft.com2medium.com3slideshare.net4docs.databricks.com5github.com6key2consulting.com

質問 # 20
A data engineer that is new to using Python needs to create a Python function to add two integers together and return the sum?
Which of the following code blocks can the data engineer use to complete this task?

正解：E

解説：
https://www.w3schools.com/python/python_functions.asp
https://www.geeksforgeeks.org/python-functions/

質問 # 21
Which of the following data lakehouse features results in improved data quality over a traditional data lake?

A. A data lakehouse provides storage solutions for structured and unstructured data.
B. A data lakehouse supports ACID-compliant transactions.
C. A data lakehouse stores data in open formats.
D. A data lakehouse allows the use of SQL queries to examine data.
E. A data lakehouse enables machine learning and artificial Intelligence workloads.

正解：B

解説：
A data lakehouse is a data management architecture that combines the flexibility, cost-efficiency, and scale of data lakes with the data management and ACID transactions of data warehouses, enabling business intelligence (BI) and machine learning (ML) on all data12. One of the key features of a data lakehouse is that it supports ACID-compliant transactions, which means that it ensures data integrity, consistency, and isolation across concurrent read and write operations3. This feature results in improved data quality over a traditional data lake, which does not support transactions and may suffer from data corruption, duplication, or inconsistency due to concurrent or streaming data ingestion and processing . References: 1: What is a Data Lakehouse? - Databricks 2: What is a Data Lakehouse? Definition, features & benefits. - Qlik 3: ACID Transactions - Databricks : [Data Lake vs Data Warehouse: Key Differences] : [Data Lakehouse: The Future of Data Engineering]

質問 # 22
A data engineer has configured a Structured Streaming job to read from a table, manipulate the data, and then perform a streaming write into a new table.
The code block used by the data engineer is below:

If the data engineer only wants the query to process all of the available data in as many batches as required, which of the following lines of code should the data engineer use to fill in the blank?

A. processingTime(1)
B. trigger(continuous="once")
C. trigger(parallelBatch=True)
D. trigger(availableNow=True)
E. trigger(processingTime="once")

正解：D

解説：
Explanation
https://stackoverflow.com/questions/71061809/trigger-availablenow-for-delta-source-streaming-queries-in-pyspa

質問 # 23
A data engineering team has noticed that their Databricks SQL queries are running too slowly when they are submitted to a non-running SQL endpoint. The data engineering team wants this issue to be resolved.
Which of the following approaches can the team use to reduce the time it takes to return results in this scenario?

A. They can turn on the Serverless feature for the SQL endpoint and change the Spot Instance Policy to
"Reliability Optimized."
B. They can increase the cluster size of the SQL endpoint.
C. They can increase the maximum bound of the SQL endpoint's scaling range
D. They can turn on the Serverless feature for the SQL endpoint.
E. They can turn on the Auto Stop feature for the SQL endpoint.

正解：D

解説：
Explanation
Databricks SQL endpoints can run in two modes: Serverless and Dedicated. Serverless mode allows you to run queries without managing clusters, while Dedicated mode allows you to run queries on a specific cluster.
Serverless mode is faster and more cost-effective for ad-hoc queries, especially when the SQL endpoint is not running. Dedicated mode is more suitable for predictable and consistent performance, especially for long-running queries. By turning on the Serverless feature for the SQL endpoint, the data engineering team can reduce the time it takes to start the SQL endpoint and return results. The other options are not relevant or effective for this scenario. References: Databricks SQL endpoints, New Performance Improvements in Databricks SQL, Slowness when fetching results in Databricks SQL

質問 # 24
A data engineer has been using a Databricks SQL dashboard to monitor the cleanliness of the input data to an ELT job. The ELT job has its Databricks SQL query that returns the number of input records containing unexpected NULL values. The data engineer wants their entire team to be notified via a messaging webhook whenever this value reaches 100.
Which of the following approaches can the data engineer use to notify their entire team via a messaging webhook whenever the number of NULL values reaches 100?

A. They can set up an Alert with a custom template.
B. They can set up an Alert with one-time notifications.
C. They can set up an Alert with a new webhook alert destination.
D. They can set up an Alert without notifications.
E. They can set up an Alert with a new email alert destination.

正解：C

解説：
A webhook alert destination is a way to send notifications to external applications or services via HTTP requests. A data engineer can use a webhook alert destination to notify their entire team via a messaging webhook, such as Slack or Microsoft Teams, whenever the number of NULL values in the input data reaches
100. To set up a webhook alert destination, the data engineer needs to do the following steps:
* In the Databricks SQL workspace, navigate to the Settings gear icon and select SQL Admin Console.
* Click Alert Destinations and click Add New Alert Destination.
* Select Webhook and enter the webhook URL and the optional custom template for the notification message.
* Click Create to save the webhook alert destination.
* In the Databricks SQL editor, create or open the query that returns the number of input records containing unexpected NULL values.
* Click the Create Alert icon above the editor window and configure the alert criteria, such as the value column, the condition, and the threshold.
* In the Notification section, select the webhook alert destination that was created earlier and click Create Alert. References: What are Databricks SQL alerts?, Monitor alerts, Monitoring Your Business with
* Alerts, Using Automation Runbook Webhooks To Alert on Databricks Status Updates.

質問 # 25
In which of the following file formats is data from Delta Lake tables primarily stored?

A. CSV
B. Parquet
C. Delta
D. A proprietary, optimized format specific to Databricks
E. JSON

正解：B

解説：
Delta Lake is an open source project that provides ACID transactions, time travel, and other features on top of Apache Parquet, a columnar file format that is widely used for big data analytics. Delta Lake uses versioned Parquet files to store your data in your cloud storage, along with JSON files as transaction logs and checkpoint files to track the changes and ensure data integrity. Delta Lake is compatible with any Apache Hive compatible file format, such as CSV, JSON, or AVRO, but it primarily stores data as Parquet files for better performance and compression. References: How to Create Delta Lake tables, 5 reasons to choose Delta Lake format (on Databricks), Parquet vs Delta format in Azure Data Lake Gen 2 store, What is Delta Lake? - Azure Databricks, Lakehouse and Delta tables - Microsoft Fabric

質問 # 26
A dataset has been defined using Delta Live Tables and includes an expectations clause:
CONSTRAINT valid_timestamp EXPECT (timestamp > '2020-01-01') ON VIOLATION DROP ROW What is the expected behavior when a batch of data containing data that violates these constraints is processed?

A. Records that violate the expectation are dropped from the target dataset and recorded as invalid in the event log.
B. Records that violate the expectation are added to the target dataset and recorded as invalid in the event log.
C. Records that violate the expectation are dropped from the target dataset and loaded into a quarantine table.
D. Records that violate the expectation are added to the target dataset and flagged as invalid in a field added to the target dataset.
E. Records that violate the expectation cause the job to fail.

正解：A

解説：
Explanation
With the defined constraint and expectation clause, when a batch of data is processed, any records that violate the expectation (in this case, where the timestamp is not greater than '2020-01-01') will be dropped from the target dataset. These dropped records will also be recorded as invalid in the event log, allowing for auditing and tracking of the data quality issues without causing the entire job to fail.
https://docs.databricks.com/en/delta-live-tables/expectations.html

質問 # 27
A data engineer only wants to execute the final block of a Python program if the Python variable day_of_week is equal to 1 and the Python variable review_period is True.
Which of the following control flow statements should the data engineer use to begin this conditionally executed code block?

A. if day_of_week = 1 and review_period:
B. if day_of_week == 1 and review_period:
C. if day_of_week == 1 and review_period == "True":
D. if day_of_week = 1 and review_period = "True":
E. if day_of_week = 1 & review_period: = "True":

正解：B

解説：
Explanation
This statement will check if the variable day_of_week is equal to 1 and if the variable review_period evaluates to a truthy value. The use of the double equal sign (==) in the comparison of day_of_week is important, as a single equal sign (=) would be used to assign a value to the variable instead of checking its value. The use of a single ampersand (&) instead of the keyword and is not valid syntax in Python. The use of quotes around True in options B and C will result in a string comparison, which will not evaluate to True even if the value of review_period is True.

質問 # 28
Which of the following statements regarding the relationship between Silver tables and Bronze tables is always true?

A. Silver tables contain aggregates while Bronze data is unaggregated.
B. Silver tables contain a more refined and cleaner view of data than Bronze tables.
C. Silver tables contain more data than Bronze tables.
D. Silver tables contain a less refined, less clean view of data than Bronze data.
E. Silver tables contain less data than Bronze tables.

正解：B

解説：
Explanation
https://www.databricks.com/glossary/medallion-architecture

質問 # 29
Which of the following commands will return the location of database customer360?

A. DROP DATABASE customer360;
B. DESCRIBE LOCATION customer360;
C. USE DATABASE customer360;
D. ALTER DATABASE customer360 SET DBPROPERTIES ('location' = '/user'};
E. DESCRIBE DATABASE customer360;

正解：E

解説：
Explanation
To retrieve the location of a database named "customer360" in a database management system like Hive or Databricks, you can use the DESCRIBE DATABASE command followed by the database name. This command will provide information about the database, including its location.

質問 # 30
A data engineer needs to apply custom logic to string column city in table stores for a specific use case. In order to apply this custom logic at scale, the data engineer wants to create a SQL user-defined function (UDF).
Which of the following code blocks creates this SQL UDF?

正解：C

解説：
Explanation
https://www.databricks.com/blog/2021/10/20/introducing-sql-user-defined-functions.html

質問 # 31
Which of the following commands can be used to write data into a Delta table while avoiding the writing of duplicate records?

A. DROP
B. INSERT
C. APPEND
D. IGNORE
E. MERGE

正解：E

解説：
The MERGE command can be used to upsert data from a source table, view, or DataFrame into a target Delta table. It allows you to specify conditions for matching and updating existing records, and inserting new records when no match is found. This way, you can avoid writing duplicate records into a Delta table1. The other commands (DROP, IGNORE, APPEND, INSERT) do not have this functionality and may result in duplicate records or data loss234. Reference: 1: Upsert into a Delta Lake table using merge | Databricks on AWS 2: SQL DELETE | Databricks on AWS 3: SQL INSERT INTO | Databricks on AWS 4: SQL UPDATE | Databricks on AWS

質問 # 32
A data engineer has been given a new record of data:
id STRING = 'a1'
rank INTEGER = 6
rating FLOAT = 9.4
Which of the following SQL commands can be used to append the new record to an existing Delta table my_table?

A. UPDATE VALUES ('a1', 6, 9.4) my_table
B. UPDATE my_table VALUES ('a1', 6, 9.4)
C. my_table UNION VALUES ('a1', 6, 9.4)
D. INSERT INTO my_table VALUES ('a1', 6, 9.4)
E. INSERT VALUES ( 'a1' , 6, 9.4) INTO my_table

正解：D

質問 # 33
Which of the following describes the type of workloads that are always compatible with Auto Loader?

A. Streaming workloads
B. Serverless workloads
C. Dashboard workloads
D. Batch workloads
E. Machine learning workloads

正解：A

解説：
Auto Loader is a Structured Streaming source that incrementally and efficiently processes new data files as they arrive in cloud storage. It supports both Python and SQL in Delta Live Tables, which are ideal for building streaming data pipelines. Auto Loader can handle near real-time ingestion of millions of files per hour and provide exactly-once guarantees when writing data into Delta Lake. Auto Loader is not designed for dashboard, machine learning, serverless, or batch workloads, which have different requirements and characteristics. Reference: What is Auto Loader?, Delta Live Tables

質問 # 34
A data engineer is designing a data pipeline. The source system generates files in a shared directory that is also used by other processes. As a result, the files should be kept as is and will accumulate in the directory. The data engineer needs to identify which files are new since the previous run in the pipeline, and set up the pipeline to only ingest those new files with each run.
Which of the following tools can the data engineer use to solve this problem?

A. Data Explorer
B. Unity Catalog
C. Auto Loader
D. Delta Lake
E. Databricks SQL

正解：C

質問 # 35
A data analyst has a series of queries in a SQL program. The data analyst wants this program to run every day. They only want the final query in the program to run on Sundays. They ask for help from the data engineering team to complete this task.
Which of the following approaches could be used by the data engineering team to complete this task?

A. They could only run the entire program on Sundays.
B. They could wrap the queries using PySpark and use Python's control flow system to determine when to run the final query.
C. They could redesign the data model to separate the data used in the final query into a new table.
D. They could submit a feature request with Databricks to add this functionality.
E. They could automatically restrict access to the source table in the final query so that it is only accessible on Sundays.

正解：B

解説：
This approach would allow the data engineering team to use the existing SQL program and add some logic to control the execution of the final query based on the day of the week. They could use the datetime module in Python to get the current date and check if it is a Sunday. If so, they could run the final query, otherwise they could skip it. This way, they could schedule the program to run every day without changing the data model or the source table. Reference: PySpark SQL Module, Python datetime Module, Databricks Jobs

質問 # 36
A data engineer that is new to using Python needs to create a Python function to add two integers together and return the sum?
Which of the following code blocks can the data engineer use to complete this task?