合格目指せDatabricks-Machine-Learning-Associate試験最新のDatabricks-Machine-Learning-Associate試験問題集PDF 2024年更新
Databricks-Machine-Learning-Associate試験問題集、365日更新無料サンプル
Databricks Databricks-Machine-Learning-Associate 認定試験の出題範囲：
| トピック | 出題範囲 |
|---|---|
| トピック 1 |
|
| トピック 2 |
|
| トピック 3 |
|
| トピック 4 |
|
質問 # 22
A data scientist has developed a linear regression model using Spark ML and computed the predictions in a Spark DataFrame preds_df with the following schema:
prediction DOUBLE
actual DOUBLE
Which of the following code blocks can be used to compute the root mean-squared-error of the model according to the data in preds_df and assign it to the rmse variable?
- A.
- B.
- C.
- D.
正解：C
解説：
The code block to compute the root mean-squared error (RMSE) for a linear regression model in Spark ML should use the RegressionEvaluator class with metricName set to "rmse". Given the schema of preds_df with columns prediction and actual, the correct evaluator setup will specify predictionCol="prediction" and labelCol="actual". Thus, the appropriate code block (Option C in your list) that uses RegressionEvaluator to compute the RMSE is the correct choice. This setup correctly measures the performance of the regression model using the predictions and actual outcomes from the DataFrame.
Reference:
Spark ML documentation (Using RegressionEvaluator to Compute RMSE).
質問 # 23
A data scientist is wanting to explore summary statistics for Spark DataFrame spark_df. The data scientist wants to see the count, mean, standard deviation, minimum, maximum, and interquartile range (IQR) for each numerical feature.
Which of the following lines of code can the data scientist run to accomplish the task?
- A. spark_df.toPandas()
- B. spark_df.printSchema()
- C. spark_df.stats()
- D. spark_df.summary ()
- E. spark_df.describe().head()
正解：D
解説：
The summary() function in PySpark's DataFrame API provides descriptive statistics which include count, mean, standard deviation, min, max, and quantiles for numeric columns. Here are the steps on how it can be used:
Import PySpark: Ensure PySpark is installed and correctly configured in the Databricks environment.
Load Data: Load the data into a Spark DataFrame.
Apply Summary: Use spark_df.summary() to generate summary statistics.
View Results: The output from the summary() function includes the statistics specified in the query (count, mean, standard deviation, min, max, and potentially quartiles which approximate the interquartile range).
Reference
PySpark Documentation: https://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.DataFrame.summary.html
質問 # 24
A data scientist is utilizing MLflow Autologging to automatically track their machine learning experiments. After completing a series of runs for the experiment experiment_id, the data scientist wants to identify the run_id of the run with the best root-mean-square error (RMSE).
Which of the following lines of code can be used to identify the run_id of the run with the best RMSE in experiment_id?
- A.
- B.
- C.
- D.
正解：B
解説：
To find the run_id of the run with the best root-mean-square error (RMSE) in an MLflow experiment, the correct line of code to use is:
mlflow.search_runs( experiment_id, order_by=["metrics.rmse"] )["run_id"][0] This line of code searches the runs in the specified experiment, orders them by the RMSE metric in ascending order (the lower the RMSE, the better), and retrieves the run_id of the best-performing run. Option C correctly represents this logic.
Reference
MLflow documentation on tracking experiments: https://www.mlflow.org/docs/latest/python_api/mlflow.html#mlflow.search_runs
質問 # 25
A data scientist wants to use Spark ML to impute missing values in their PySpark DataFrame features_df. They want to replace missing values in all numeric columns in features_df with each respective numeric column's median value.
They have developed the following code block to accomplish this task:
The code block is not accomplishing the task.
Which reasons describes why the code block is not accomplishing the imputation task?
- A. The fit method needs to be called instead of transform.
- B. It does not fit the imputer on the data to create an ImputerModel.
- C. It does not impute both the training and test data sets.
- D. The inputCols and outputCols need to be exactly the same.
正解：B
解説：
In the provided code block, the Imputer object is created but not fitted on the data to generate an ImputerModel. The transform method is being called directly on the Imputer object, which does not yet contain the fitted median values needed for imputation. The correct approach is to fit the imputer on the dataset first.
Corrected code:
imputer = Imputer( strategy="median", inputCols=input_columns, outputCols=output_columns ) imputer_model = imputer.fit(features_df) # Fit the imputer to the data imputed_features_df = imputer_model.transform(features_df) # Transform the data using the fitted imputer Reference:
PySpark ML Documentation
質問 # 26
A data scientist is developing a machine learning pipeline using AutoML on Databricks Machine Learning.
Which of the following steps will the data scientist need to perform outside of their AutoML experiment?
- A. Exploratory data analysis
- B. Model tuning
- C. Model deployment
- D. Model evaluation
正解：A
解説：
AutoML platforms, such as the one available in Databricks Machine Learning, streamline various stages of the machine learning pipeline including feature engineering, model selection, hyperparameter tuning, and model evaluation. However, exploratory data analysis (EDA) is typically performed outside the AutoML process. EDA involves understanding the dataset, visualizing distributions, identifying anomalies, and gaining insights into data before feeding it into a machine learning pipeline. This step is crucial for ensuring that the data is clean and suitable for model training but is generally done manually by the data scientist.
Reference
Databricks documentation on AutoML: https://docs.databricks.com/applications/machine-learning/automl.html
質問 # 27
A data scientist has been given an incomplete notebook from the data engineering team. The notebook uses a Spark DataFrame spark_df on which the data scientist needs to perform further feature engineering. Unfortunately, the data scientist has not yet learned the PySpark DataFrame API.
Which of the following blocks of code can the data scientist run to be able to use the pandas API on Spark?
- A. import pyspark.pandas as ps
df = ps.DataFrame(spark_df) - B. spark_df.to_sql()
- C. spark_df.to_pandas()
- D. import pyspark.pandas as ps
df = ps.to_pandas(spark_df) - E. import pandas as pd
df = pd.DataFrame(spark_df)
正解：A
解説：
To use the pandas API on Spark, which is designed to bridge the gap between the simplicity of pandas and the scalability of Spark, the correct approach involves importing the pyspark.pandas (recently renamed to pandas_api_on_spark) module and converting a Spark DataFrame to a pandas-on-Spark DataFrame using this API. The provided syntax correctly initializes a pandas-on-Spark DataFrame, allowing the data scientist to work with the familiar pandas-like API on large datasets managed by Spark.
Reference
Pandas API on Spark Documentation: https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/index.html
質問 # 28
A machine learning engineer has created a Feature Table new_table using Feature Store Client fs. When creating the table, they specified a metadata description with key information about the Feature Table. They now want to retrieve that metadata programmatically.
Which of the following lines of code will return the metadata description?
- A. fs.get_table("new_table").description
- B. fs.get_table("new_table").load_df()
- C. fs.get_table("new_table")
- D. fs.create_training_set("new_table")
- E. There is no way to return the metadata description programmatically.
正解：A
解説：
To retrieve the metadata description of a feature table created using the Feature Store Client (referred here as fs), the correct method involves calling get_table on the fs client with the table name as an argument, followed by accessing the description attribute of the returned object. The code snippet fs.get_table("new_table").description correctly achieves this by fetching the table object for "new_table" and then accessing its description attribute, where the metadata is stored. The other options do not correctly focus on retrieving the metadata description.
Reference:
Databricks Feature Store documentation (Accessing Feature Table Metadata).
質問 # 29
The implementation of linear regression in Spark ML first attempts to solve the linear regression problem using matrix decomposition, but this method does not scale well to large datasets with a large number of variables.
Which of the following approaches does Spark ML use to distribute the training of a linear regression model for large data?
- A. Singular value decomposition
- B. Logistic regression
- C. Spark ML cannot distribute linear regression training
- D. Least-squares method
- E. Iterative optimization
正解：E
質問 # 30
A machine learning engineer is trying to perform batch model inference. They want to get predictions using the linear regression model saved at the path model_uri for the DataFrame batch_df.
batch_df has the following schema:
customer_id STRING
The machine learning engineer runs the following code block to perform inference on batch_df using the linear regression model at model_uri:
In which situation will the machine learning engineer's code block perform the desired inference?
- A. When the Feature Store feature set was logged with the model at model_uri
- B. This code block will not perform the desired inference in any situation.
- C. When all of the features used by the model at model_uri are in a Spark DataFrame in the PySpark
- D. When all of the features used by the model at model_uri are in a single Feature Store table
- E. When the model at model_uri only uses customer_id as a feature
正解：A
解説：
The code block provided by the machine learning engineer will perform the desired inference when the Feature Store feature set was logged with the model at model_uri. This ensures that all necessary feature transformations and metadata are available for the model to make predictions. The Feature Store in Databricks allows for seamless integration of features and models, ensuring that the required features are correctly used during inference.
Reference:
Databricks documentation on Feature Store: Feature Store in Databricks
質問 # 31
A data scientist is attempting to tune a logistic regression model logistic using scikit-learn. They want to specify a search space for two hyperparameters and let the tuning process randomly select values for each evaluation.
They attempt to run the following code block, but it does not accomplish the desired task:
Which of the following changes can the data scientist make to accomplish the task?
- A. Replace the penalty= ['12', '11'] argument with penalty=uniform ('12', '11')
- B. Replace the GridSearchCV operation with ParameterGrid
- C. Replace the GridSearchCV operation with cross_validate
- D. Replace the random_state=0 argument with random_state=1
- E. Replace the GridSearchCV operation with RandomizedSearchCV
正解：E
解説：
The user wants to specify a search space for hyperparameters and let the tuning process randomly select values. GridSearchCV systematically tries every combination of the provided hyperparameter values, which can be computationally expensive and time-consuming. RandomizedSearchCV, on the other hand, samples hyperparameters from a distribution for a fixed number of iterations. This approach is usually faster and still can find very good parameters, especially when the search space is large or includes distributions.
Reference
Scikit-Learn documentation on hyperparameter tuning: https://scikit-learn.org/stable/modules/grid_search.html#randomized-parameter-optimization
質問 # 32
What is the name of the method that transforms categorical features into a series of binary indicator feature variables?
- A. Leave-one-out encoding
- B. String indexing
- C. Target encoding
- D. One-hot encoding
- E. Categorical
正解：D
解説：
The method that transforms categorical features into a series of binary indicator variables is known as one-hot encoding. This technique converts each categorical value into a new binary column, which is essential for models that require numerical input. One-hot encoding is widely used because it helps to handle categorical data without introducing a false ordinal relationship among categories.
Reference:
Feature Engineering Techniques (One-Hot Encoding).
質問 # 33
A data scientist learned during their training to always use 5-fold cross-validation in their model development workflow. A colleague suggests that there are cases where a train-validation split could be preferred over k-fold cross-validation when k > 2.
Which of the following describes a potential benefit of using a train-validation split over k-fold cross-validation in this scenario?
- A. A holdout set is not necessary when using a train-validation split
- B. Bias is avoidable when using a train-validation split
- C. Fewer hyperparameter values need to be tested when using a train-validation split
- D. Reproducibility is achievable when using a train-validation split
- E. Fewer models need to be trained when using a train-validation split
正解：E
解説：
A train-validation split is often preferred over k-fold cross-validation (with k > 2) when computational efficiency is a concern. With a train-validation split, only two models (one on the training set and one on the validation set) are trained, whereas k-fold cross-validation requires training k models (one for each fold).
This reduction in the number of models trained can save significant computational resources and time, especially when dealing with large datasets or complex models.
Reference:
Model Evaluation with Train-Test Split
質問 # 34
A data scientist learned during their training to always use 5-fold cross-validation in their model development workflow. A colleague suggests that there are cases where a train-validation split could be preferred over k-fold cross-validation when k > 2.
Which of the following describes a potential benefit of using a train-validation split over k-fold cross-validation in this scenario?
- A. A holdout set is not necessary when using a train-validation split
- B. Bias is avoidable when using a train-validation split
- C. Fewer hyperparameter values need to be tested when using a train-validation split
- D. Reproducibility is achievable when using a train-validation split
- E. Fewer models need to be trained when using a train-validation split
正解：E
質問 # 35
A team is developing guidelines on when to use various evaluation metrics for classification problems. The team needs to provide input on when to use the F1 score over accuracy.
Which of the following suggestions should the team include in their guidelines?
- A. The F1 score should be utilized over accuracy when the number of actual positive cases is identical to the number of actual negative cases.
- B. The F1 score should be utilized over accuracy when there is significant imbalance between positive and negative classes and avoiding false negatives is a priority.
- C. The F1 score should be utilized over accuracy when there are greater than two classes in the target variable.
- D. The F1 score should be utilized over accuracy when identifying true positives and true negatives are equally important to the business problem.
正解：B
解説：
The F1 score is the harmonic mean of precision and recall and is particularly useful in situations where there is a significant imbalance between positive and negative classes. When there is a class imbalance, accuracy can be misleading because a model can achieve high accuracy by simply predicting the majority class. The F1 score, however, provides a better measure of the test's accuracy in terms of both false positives and false negatives.
Specifically, the F1 score should be used over accuracy when:
There is a significant imbalance between positive and negative classes.
Avoiding false negatives is a priority, meaning recall (the ability to detect all positive instances) is crucial.
In this scenario, the F1 score balances both precision (the ability to avoid false positives) and recall, providing a more meaningful measure of a model's performance under these conditions.
Reference:
Databricks documentation on classification metrics: Classification Metrics
質問 # 36
An organization is developing a feature repository and is electing to one-hot encode all categorical feature variables. A data scientist suggests that the categorical feature variables should not be one-hot encoded within the feature repository.
Which of the following explanations justifies this suggestion?
- A. One-hot encoding is computationally intensive and should only be performed on small samples of training sets for individual machine learning problems.
- B. One-hot encoding is a potentially problematic categorical variable strategy for some machine learning algorithms.
- C. One-hot encoding is dependent on the target variable's values which differ for each apaplication.
- D. One-hot encoding is not a common strategy for representing categorical feature variables numerically.
正解：B
解説：
The suggestion not to one-hot encode categorical feature variables within the feature repository is justified because one-hot encoding can be problematic for some machine learning algorithms. Specifically, one-hot encoding increases the dimensionality of the data, which can be computationally expensive and may lead to issues such as multicollinearity and overfitting. Additionally, some algorithms, such as tree-based methods, can handle categorical variables directly without requiring one-hot encoding.
Reference:
Databricks documentation on feature engineering: Feature Engineering
質問 # 37
A machine learning engineer would like to develop a linear regression model with Spark ML to predict the price of a hotel room. They are using the Spark DataFrame train_df to train the model.
The Spark DataFrame train_df has the following schema:
The machine learning engineer shares the following code block:
Which of the following changes does the machine learning engineer need to make to complete the task?
- A. They need to call the transform method on train df
- B. They need to convert the features column to be a vector
- C. They need to utilize a Pipeline to fit the model
- D. They need to split the features column out into one column for each feature
- E. They do not need to make any changes
正解：B
解説：
In Spark ML, the linear regression model expects the feature column to be a vector type. However, if the features column in the DataFrame train_df is not already in this format (such as being a column of type UDT or a non-vectorized type), the engineer needs to convert it to a vector column using a transformer like VectorAssembler. This is a critical step in preparing the data for modeling as Spark ML models require input features to be combined into a single vector column.
Reference
Spark MLlib documentation for LinearRegression: https://spark.apache.org/docs/latest/ml-classification-regression.html#linear-regression
質問 # 38
Which of the following is a benefit of using vectorized pandas UDFs instead of standard PySpark UDFs?
- A. The vectorized pandas UDFs allow for the use of type hints
- B. The vectorized pandas UDFs process data in memory rather than spilling to disk
- C. The vectorized pandas UDFs allow for pandas API use inside of the function
- D. The vectorized pandas UDFs work on distributed DataFrames
- E. The vectorized pandas UDFs process data in batches rather than one row at a time
正解：E
解説：
Vectorized pandas UDFs, also known as Pandas UDFs, are a powerful feature in PySpark that allows for more efficient operations than standard UDFs. They operate by processing data in batches, utilizing vectorized operations that leverage pandas to perform operations on whole batches of data at once. This approach is much more efficient than processing data row by row as is typical with standard PySpark UDFs, which can significantly speed up the computation.
Reference
PySpark Documentation on UDFs: https://spark.apache.org/docs/latest/api/python/user_guide/sql/arrow_pandas.html#pandas-udfs-a-k-a-vectorized-udfs
質問 # 39
......
Databricks-Machine-Learning-Associate問題集、あなたを合格させる認証試験：https://jp.fast2test.com/Databricks-Machine-Learning-Associate-premium-file.html