2024年最新の有効なProfessional-Data-Engineerリアル試験問題（更新された）100%問題集と練習試験合格させます
[更新されたのは2024年]Google Professional-Data-Engineer問題準備には無料サンプルのPDF
質問 # 204
You need to store and analyze social media postings in Google BigQuery at a rate of 10,000 messages per minute in near real-time. Initially, design the application to use streaming inserts for individual postings. Your application also performs data aggregations right after the streaming inserts. You discover that the queries after streaming inserts do not exhibit strong consistency, and reports from the queries might miss in-flight data. How can you adjust your application design?
- A. Re-write the application to load accumulated data every 2 minutes.
- B. Load the original message to Google Cloud SQL, and export the table every hour to BigQuery via streaming inserts.
- C. Convert the streaming insert code to batch load for individual messages.
- D. Estimate the average latency for data availability after streaming inserts, and always run queries after waiting twice as long.
正解：D
解説：
Explanation
The data is first comes to buffer and then written to Storage. If we are running queries in buffer we will face above mentioned issues. If we wait for the bigquery to write the data to storage then we won't face the issue.
So We need to wait till it's written tio storage
質問 # 205
You are using BigQuery with a regional dataset that includes a table with the daily sales volumes. This table is updated multiple times per day. You need to protect your sales table in case of regional failures with a recovery point objective (RPO) of less than 24 hours, while keeping costs to a minimum. What should you do?
- A. Modify ETL job to load the data into both the current and another backup region.
- B. Schedule a daily copy of the dataset to a backup region.
- C. Schedule a daily export of the table to a Cloud Storage dual or multi-region bucket.
- D. Schedule a daily BigQuery snapshot of the table.
正解：D
解説：
To apply complex business logic on a JSON response using Python's standard library within a Workflow, invoking a Cloud Function is the most efficient and straightforward approach. Here's why option A is the best choice:
Cloud Functions:
Cloud Functions provide a lightweight, serverless execution environment for running code in response to events. They support Python and can easily integrate with Workflows.
This approach ensures simplicity and speed of execution, as Cloud Functions can be invoked directly from a Workflow and handle the complex logic required.
Flexibility and Simplicity:
Using Cloud Functions allows you to leverage Python's extensive standard library and ecosystem, making it easier to implement and maintain the complex business logic.
Cloud Functions abstract the underlying infrastructure, allowing you to focus on the application logic without worrying about server management.
Performance:
Cloud Functions are optimized for fast execution and can handle the processing of the JSON response efficiently.
They are designed to scale automatically based on demand, ensuring that your workflow remains performant.
Steps to Implement:
Write the Cloud Function:
Develop a Cloud Function in Python that processes the JSON response and applies the necessary business logic.
Deploy the function to Google Cloud.
Invoke Cloud Function from Workflow:
Modify your Workflow to call the Cloud Function using an HTTP request or Google Cloud Function connector.
steps:
- callCloudFunction:
call: http.post
args:
url: https://REGION-PROJECT_ID.cloudfunctions.net/FUNCTION_NAME
body:
key: value
Process Results:
Handle the response from the Cloud Function and proceed with the next steps in the Workflow, such as loading data into BigQuery.
Reference:
Google Cloud Functions Documentation
Using Workflows with Cloud Functions
Workflows Standard Library
質問 # 206
You migrated your on-premises Apache Hadoop Distributed File System (HDFS) data lake to Cloud Storage. The data scientist team needs to process the data by using Apache Spark and SQL. Security policies need to be enforced at the column level. You need a cost-effective solution that can scale into a data mesh. What should you do?
- A. 1 Apply an Identity and Access Management (IAM) policy at the file level in Cloud Storage
2. Define a BigQuery external table for SQL processing.
3. Use Dataproc Spark to process the Cloud Storage files. - B. 1. Deploy a long-living Dalaproc cluster with Apache Hive and Ranger enabled.
2. Configure Ranger for column level security.
3. Process with Dataproc Spark or Hive SQL. - C. 1. Define a BigLake table.
2. Create a taxonomy of policy tags in Data Catalog.
3. Add policy lags to columns.
4. Process with the Spark-BigQuery connector or BigQuery SOL. - D. 1. Load the data to BigQuery tables.
2. Create a taxonomy of policy tags in Data Catalog.
3. Add policy tags to columns.
4. Procoss with the Spark-BigQuery connector or BigQuery SQL.
正解：A
解説：
For automating the CI/CD pipeline of DAGs running in Cloud Composer, the following approach ensures that DAGs are tested and deployed in a streamlined and efficient manner.
Use Cloud Build for Development Instance Testing:
Use Cloud Build to automate the process of copying the DAG code to the Cloud Storage bucket of the development instance.
This triggers Cloud Composer to automatically pick up and test the new DAGs in the development environment.
Testing and Validation:
Ensure that the DAGs run successfully in the development environment.
Validate the functionality and correctness of the DAGs before promoting them to production.
Deploy to Production:
If the DAGs pass all tests in the development environment, use Cloud Build to copy the tested DAG code to the Cloud Storage bucket of the production instance.
This ensures that only validated and tested DAGs are deployed to production, maintaining the stability and reliability of the production environment.
Simplicity and Reliability:
This approach leverages Cloud Build's capabilities for automation and integrates seamlessly with Cloud Composer's reliance on Cloud Storage for DAG storage.
By using Cloud Storage for both development and production deployments, the process remains simple and robust.
Google Data Engineer Reference:
Cloud Composer Documentation
Using Cloud Build
Deploying DAGs to Cloud Composer
Automating DAG Deployment with Cloud Build
By implementing this CI/CD pipeline, you ensure that DAGs are thoroughly tested in the development environment before being automatically deployed to the production environment, maintaining high quality and reliability.
質問 # 207
How can you get a neural network to learn about relationships between categories in a categorical feature?
- A. Create a multi-hot column
- B. Create a hash bucket
- C. Create a one-hot column
- D. Create an embedding column
正解：D
解説：
There are two problems with one-hot encoding. First, it has high dimensionality, meaning that instead of having just one value, like a continuous feature, it has many values, or dimensions. This makes computation more time-consuming, especially if a feature has a very large number of categories. The second problem is that it doesn't encode any relationships between the categories. They are completely independent from each other, so the network has no way of knowing which ones are similar to each other.
Both of these problems can be solved by representing a categorical feature with an embedding column. The idea is that each category has a smaller vector with, let's say, 5 values in it. But unlike a one-hot vector, the values are not usually 0. The values are weights, similar to the weights that are used for basic features in a neural network. The difference is that each category has a set of weights (5 of them in this case).
You can think of each value in the embedding vector as a feature of the category. So, if two categories are very similar to each other, then their embedding vectors should be very similar too.
Reference:
https://cloudacademy.com/google/introduction-to-google-cloud-machine-learning-engine-course/a-wide-and-dee
質問 # 208
Which of the following is NOT true about Dataflow pipelines?
- A. Dataflow pipelines can be programmed in Java
- B. Dataflow pipelines are tied to Dataflow, and cannot be run on any other runner
- C. Dataflow pipelines use a unified programming model, so can work both with streaming and batch data sources
- D. Dataflow pipelines can consume data from other Google Cloud services
正解：B
解説：
Explanation
Dataflow pipelines can also run on alternate runtimes like Spark and Flink, as they are built using the Apache Beam SDKs Reference: https://cloud.google.com/dataflow/
質問 # 209
You create an important report for your large team in Google Data Studio 360. The report uses Google
BigQuery as its data source. You notice that visualizations are not showing data that is less than 1 hour
old. What should you do?
- A. Clear your browser history for the past hour then reload the tab showing the virtualizations.
- B. Refresh your browser tab showing the visualizations.
- C. Disable caching by editing the report settings.
- D. Disable caching in BigQuery by editing table details.
正解：C
解説：
Explanation/Reference:
Reference: https://support.google.com/datastudio/answer/7020039?hl=en
質問 # 210
You work for a manufacturing company that sources up to 750 different components, each from a different supplier. You've collected a labeled dataset that has on average 1000 examples for each unique component.
Your team wants to implement an app to help warehouse workers recognize incoming components based on a photo of the component. You want to implement the first working version of this app (as Proof-Of-Concept) within a few working days. What should you do?
- A. Use Cloud Vision AutoML with the existing dataset.
- B. Train your own image recognition model leveraging transfer learning techniques.
- C. Use Cloud Vision API by providing custom labels as recognition hints.
- D. Use Cloud Vision AutoML, but reduce your dataset twice.
正解：A
質問 # 211
Your globally distributed auction application allows users to bid on items. Occasionally, users place identical bids at nearly identical times, and different application servers process those bids. Each bid event contains the item, amount, user, and timestamp. You want to collate those bid events into a single location in real time to determine which user bid first. What should you do?
- A. Create a file on a shared file and have the application servers write all bid events to that file. Process the file with Apache Hadoop to identify which user bid first.
- B. Set up a MySQL database for each application server to write bid events into. Periodically query each of those distributed MySQL databases and update a master MySQL database with bid event information.
- C. Have each application server write the bid events to Cloud Pub/Sub as they occur. Push the events from Cloud Pub/Sub to a custom endpoint that writes the bid event information into Cloud SQL.
- D. Have each application server write the bid events to Google Cloud Pub/Sub as they occur. Use a pull
正解：B
解説：
subscription to pull the bid events using Google Cloud Dataflow. Give the bid for each item to the user in the bid event that is processed first.
質問 # 212
You have enabled the free integration between Firebase Analytics and Google BigQuery. Firebase now
automatically creates a new table daily in BigQuery in the format app_events_YYYYMMDD. You want to
query all of the tables for the past 30 days in legacy SQL. What should you do?
- A. Use the TABLE_DATE_RANGE function
- B. Use WHERE date BETWEEN YYYY-MM-DD AND YYYY-MM-DD
- C. Use SELECT IF.(date >= YYYY-MM-DD AND date <= YYYY-MM-DD
- D. Use the WHERE_PARTITIONTIME pseudo column
正解：A
質問 # 213
You are on the data governance team and are implementing security requirements to deploy resources. You need to ensure that resources are limited to only the europe-west 3 region You want to follow Google-recommended practices What should you do?
- A. Set the constraints/gcp. resourceLocations organization policy constraint to in: europe-west3-locations.
- B. Create a Cloud Function to monitor all resources created and automatically destroy the ones created outside the europe-west3 region.
- C. Set the constraints/gcp. resourceLocations organization policy constraint to in:eu-locations.
- D. Deploy resources with Terraform and implement a variable validation rule to ensure that the region is set to the europe-west3 region for all resources.
正解：A
解説：
To ensure that resources are limited to only the europe-west3 region, you should set the organization policy constraint constraints/gcp.resourceLocations to in:europe-west3-locations. This policy restricts the deployment of resources to the specified locations, which in this case is the europe-west3 region. By setting this policy, you enforce location compliance across your Google Cloud resources, aligning with the best practices for data governance and regulatory compliance.
Reference:
Professional Data Engineer Certification Exam Guide | Learn - Google Cloud1.
Preparing for Google Cloud Certification: Cloud Data Engineer2.
Professional Data Engineer Certification | Learn | Google Cloud3.
3: Professional Data Engineer Certification | Learn | Google Cloud 2: Preparing for Google Cloud Certification: Cloud Data Engineer 1: Professional Data Engineer Certification Exam Guide | Learn - Google Cloud
質問 # 214
You want to use a database of information about tissue samples to classify future tissue samples as either normal or mutated. You are evaluating an unsupervised anomaly detection method for classifying the tissue samples. Which two characteristic support this method? (Choose two.)
- A. You expect future mutations to have similar features to the mutated samples in the database.
- B. You already have labels for which samples are mutated and which are normal in the database.
- C. There are very few occurrences of mutations relative to normal samples.
- D. You expect future mutations to have different features from the mutated samples in the database.
- E. There are roughly equal occurrences of both normal and mutated samples in the database.
正解：D、E
質問 # 215
When you design a Google Cloud Bigtable schema it is recommended that you _________.
- A. Avoid schema designs that require atomicity across rows
- B. Create schema designs that are based on a relational database design
- C. Create schema designs that require atomicity across rows
- D. Avoid schema designs that are based on NoSQL concepts
正解：A
解説：
All operations are atomic at the row level. For example, if you update two rows in a table, it's possible that one row will be updated successfully and the other update will fail. Avoid schema designs that require atomicity across rows.
質問 # 216
You are building a data pipeline on Google Cloud. You need to prepare data using a casual method for a machine-learning process. You want to support a logistic regression model. You also need to monitor and adjust for null values, which must remain real-valued and cannot be removed. What should you do?
- A. Use Cloud Dataprep to find null values in sample source data. Convert all nulls to 0 using a Cloud Dataprep job.
- B. Use Cloud Dataflow to find null values in sample source data. Convert all nulls to using a custom script.
- C. Use Cloud Dataflow to find null values in sample source data. Convert all nulls to `none' using a Cloud Dataprep job.
- D. Use Cloud Dataprep to find null values in sample source data. Convert all nulls to `none' using a Cloud Dataproc job.
正解：C
質問 # 217
You have a job that you want to cancel. It is a streaming pipeline, and you want to ensure that any data that is in-flight is processed and written to the output. Which of the following commands can you use on the Dataflow monitoring console to stop the pipeline job?
- A. Drain
- B. Cancel
- C. Finish
- D. Stop
正解：A
解説：
Explanation
Using the Drain option to stop your job tells the Dataflow service to finish your job in its current state. Your job will immediately stop ingesting new data from input sources, but the Dataflow service will preserve any existing resources (such as worker instances) to finish processing and writing any buffered data in your pipeline.
Reference: https://cloud.google.com/dataflow/pipelines/stopping-a-pipeline
質問 # 218
You need to create a new transaction table in Cloud Spanner that stores product sales data. You are deciding what to use as a primary key. From a performance perspective, which strategy should you choose?
- A. The original order identification number from the sales system, which is a monotonically increasing integer
- B. A concatenation of the product name and the current epoch time
- C. A random universally unique identifier number (version 4 UUID)
- D. The current epoch time
正解：C
解説：
Explanation/Reference: https://www.uuidgenerator.net/version4
質問 # 219
Business owners at your company have given you a database of bank transactions. Each row contains the user ID, transaction type, transaction location, and transaction amount. They ask you to investigate what type of machine learning can be applied to the data. Which three machine learning applications can you use? (Choose three.)
- A. Reinforcement learning to predict the location of a transaction.
- B. Clustering to divide the transactions into N categories based on feature similarity.
- C. Unsupervised learning to predict the location of a transaction.
- D. Unsupervised learning to determine which transactions are most likely to be fraudulent.
- E. Supervised learning to determine which transactions are most likely to be fraudulent.
- F. Supervised learning to predict the location of a transaction.
正解：A、B、D
質問 # 220
You are using Google BigQuery as your data warehouse. Your users report that the following simple query is running very slowly, no matter when they run the query:
SELECT country, state, city FROM [myproject:mydataset.mytable] GROUP BY country You check the query plan for the query and see the following output in the Read section of Stage:1:
What is the most likely cause of the delay for this query?
- A. Either the state or the city columns in the [myproject:mydataset.mytable] table have too many NULL values
- B. The [myproject:mydataset.mytable] table has too many partitions
- C. Users are running too many concurrent queries in the system
- D. Most rows in the [myproject:mydataset.mytable] table have the same value in the country column, causing data skew
正解：D
質問 # 221
Your company needs to upload their historic data to Cloud Storage. The security rules don't allow access from external IPs to their on-premises resources. After an initial upload, they will add new data from existing on-premises applications every day. What should they do?
- A. Write a job template in Cloud Dataproc to perform the data transfer.
- B. Execute gsutil rsync from the on-premises servers.
- C. Use Cloud Dataflow and write the data to Cloud Storage.
- D. Install an FTP server on a Compute Engine VM to receive the files and move them to Cloud Storage.
正解：B
質問 # 222
Your company built a TensorFlow neural-network model with a large number of neurons and layers. The model fits well for the training data. However, when tested against new data, it performs poorly. What method can you employ to address this?
- A. Dimensionality Reduction
- B. Dropout Methods
- C. Serialization
- D. Threading
正解：B
解説：
Reference
https://medium.com/mlreview/a-simple-deep-learning-model-for-stock-price-prediction-using-tensorflow-30505
質問 # 223
Your startup has never implemented a formal security policy. Currently, everyone in the company has access to the datasets stored in Google BigQuery. Teams have freedom to use the service as they see fit, and they have not documented their use cases. You have been asked to secure the data warehouse. You need to discover what everyone is doing. What should you do first?
- A. Use Stackdriver Monitoring to see the usage of BigQuery query slots.
- B. Get the identity and access management IIAM) policy of each table
- C. Use the Google Cloud Billing API to see what account the warehouse is being billed to.
- D. Use Google Stackdriver Audit Logs to review data access.
正解：A
質問 # 224
You are designing a cloud-native historical data processing system to meet the following conditions:
* The data being analyzed is in CSV, Avro, and PDF formats and will be accessed by multiple analysis tools including Cloud Dataproc, BigQuery, and Compute Engine.
* A streaming data pipeline stores new data daily.
* Peformance is not a factor in the solution.
* The solution design should maximize availability.
How should you design data storage for this solution?
- A. Store the data in a multi-regional Cloud Storage bucket. Access the data directly using Cloud Dataproc, BigQuery, and Compute Engine.
- B. Store the data in a regional Cloud Storage bucket. Aceess the bucket directly using Cloud Dataproc, BigQuery, and Compute Engine.
- C. Store the data in BigQuery. Access the data using the BigQuery Connector or Cloud Dataproc and Compute Engine.
- D. Create a Cloud Dataproc cluster with high availability. Store the data in HDFS, and peform analysis as needed.
正解：A
質問 # 225
How can you get a neural network to learn about relationships between categories in a categorical feature?
- A. Create a multi-hot column
- B. Create a hash bucket
- C. Create a one-hot column
- D. Create an embedding column
正解：D
解説：
There are two problems with one-hot encoding. First, it has high dimensionality, meaning that instead of having just one value, like a continuous feature, it has many values, or dimensions. This makes computation more time-consuming, especially if a feature has a very large number of categories. The second problem is that it doesn't encode any relationships between the categories. They are completely independent from each other, so the network has no way of knowing which ones are similar to each other.
Both of these problems can be solved by representing a categorical feature with an embedding column. The idea is that each category has a smaller vector with, let's say, 5 values in it. But unlike a one-hot vector, the values are not usually 0. The values are weights, similar to the weights that are used for basic features in a neural network. The difference is that each category has a set of weights (5 of them in this case).
You can think of each value in the embedding vector as a feature of the category. So, if two categories are very similar to each other, then their embedding vectors should be very similar too.
質問 # 226
Your company is implementing a data warehouse using BigQuery, and you have been tasked with designing the data model You move your on-premises sales data warehouse with a star data schema to BigQuery but notice performance issues when querying the data of the past 30 days Based on Google's recommended practices, what should you do to speed up the query without increasing storage costs?
- A. Materialize the dimensional data in views
- B. Shard the data by customer ID
- C. Denormalize the data
- D. Partition the data by transaction date
正解：A
質問 # 227
......
Professional-Data-Engineer豪華セット学習ガイドにはオンライン試験エンジン：https://jp.fast2test.com/Professional-Data-Engineer-premium-file.html
2024年最新の認定サンプル問題Professional-Data-Engineer問題集と練習試験：https://drive.google.com/open?id=1cIC1E7akzgrHSRItnvyUvBQ7MpVo8LJD