Amazon Practice Questions, Discussions & Exam Topics by our Authors
A data engineer wants to improve the performance of SQL queries in Amazon Athena that run against a sales data table.The data engineer wants to understand the execution plan of a specific SQL statement. The data engineer also wants to see the computational cost...
Let's analyze each option and how it meets the requirements of understanding the execution plan and computational cost for SQL queries in Amazon Athena:
A) EXPLAIN SELECT FROM sales;
- Why it's not ideal: The `EXPLAIN` statement provides the query execution plan, which outlines the steps Athena will take to execute the query, such as scanning tables, joining data, and applying filters. However, it doesn't provide detailed runtime statistics or computational cost for each operation. It's useful for analyzing the query structure, but it doesn't provide the full performance analysis that includes costs.
- Best scenario for this: This option would work if the goal is to simply view the query execution plan, but it doesn't give the computational cost or performance details.
B) EXPLAIN ANALYZE FROM sales;
- Why it's not ideal: The statement `EXPLAIN ANALYZE` without a `SELECT` query is not syntactically correct for Athena. In Amazon Athena, you need to specify a `SELECT` query after the `EXPLAIN ANALYZE` statement to get detailed performance metrics, including computational costs. As it stands, this statement would result in an error.
- Best scenario for this: This is not a valid option because of incorrect syntax in Athena.
C) EXPL...
Author: Michael · Last updated Jul 30, 2026
A company plans to provision a log delivery stream within a VPC. The company configured the VPC flow logs to publish to Amazon CloudWatch Logs. The company needs to send the flow logs to Splunk in near real time for fur...
Let's evaluate each option based on the requirement to send VPC flow logs to Splunk in near real-time with the least operational overhead.
A) Configure an Amazon Kinesis Data Streams data stream to use Splunk as the destination. Create a CloudWatch Logs subscription filter to send log events to the data stream.
- Why it’s not ideal: While Kinesis Data Streams is a good option for real-time data streaming, it requires a custom application or service to consume the data from the stream and send it to Splunk. Configuring a subscription filter to send CloudWatch Logs to Kinesis Data Streams adds complexity, as you'd need to build and maintain the integration between Kinesis and Splunk. This introduces additional operational overhead and management complexity.
- Best scenario for this: This would work in scenarios where you need full control over the data flow and processing pipeline but is more complex and requires more maintenance.
B) Create an Amazon Kinesis Data Firehose delivery stream to use Splunk as the destination. Create a CloudWatch Logs subscription filter to send log events to the delivery stream.
- Why it’s ideal: Kinesis Data Firehose simplifies the process by providing a fully managed service that can stream data directly to Splunk. It handles data delivery and transformation automatically, which reduces operational overhead. The CloudWatch Logs subscription filter can be easily configured to send log events to the Firehose delivery stream. This approach is highly streamlined, reducing the need for custom applications or additional components.
- Best scenario for this: This option is the best choice when you want to minimize operational overhead and take advantage of a managed service that integrates well with CloudWatch Logs and Splunk. It is the easiest way to deliver data to Splunk in near real-time without additional maintenance.
C) Create an Amazon Kinesis Data Firehose delivery stream to use Splunk...
Author: Stella · Last updated Jul 30, 2026
A company has a data lake on AWS. The data lake ingests sources of data from business units. The company uses Amazon Athena for queries. The storage layer is Amazon S3 with an AWS Glue Data Catalog as a metadata repository.The company wants to make the data available to data scientists and business analysts. However, the company first needs t...
Key Factors:
- Fine-grained access control: The company wants to ensure column-level access control based on user roles and responsibilities.
- Integration with Athena: The solution needs to work seamlessly with Amazon Athena, which queries data stored in S3 and uses the AWS Glue Data Catalog as a metadata repository.
- User roles and responsibilities: The solution must account for managing different access levels based on user roles, which may include data scientists, business analysts, and others.
Option Analysis:
A) Set up AWS Lake Formation. Define security policy-based rules for the users and applications by IAM role in Lake Formation.
- Pros:
- AWS Lake Formation is specifically designed for data lakes, offering fine-grained, column-level access control for data stored in Amazon S3 and cataloged in AWS Glue.
- It allows you to define policies for users or applications based on IAM roles.
- It integrates well with Athena for secure querying of the data.
- It provides a centralized mechanism for managing access controls, including fine-grained permissions (e.g., column-level access) to the data.
- Cons:
- Requires an extra layer of configuration compared to other options (though it is highly effective for managing data lake security).
- Use Case: This option is ideal for companies using AWS Glue and Athena with S3 for a data lake, especially when you need fine-grained access controls based on specific data access needs.
B) Define an IAM resource-based policy for AWS Glue tables. Attach the same policy to IAM user groups.
- Pros:
- IAM resource-based policies can control access to AWS Glue resources, including tables.
- Cons:
- IAM policies generally do not support column-level access control in a fine-grained manner, and they are more suited for broader permission sets rather than detailed access management.
- This option doesn't meet the column-level access requirement needed in the scenario.
- Use Case: T...
Author: MoonlitPantherX · Last updated Jul 30, 2026
A company has developed several AWS Glue extract, transform, and load (ETL) jobs to validate and transform data from Amazon S3. The ETL jobs load the data into Amazon RDS for MySQL in batches once every day. The ETL jobs use a DynamicFrame to read the S3 data.The ETL jobs currently process all the data that is in the S3 bucket. Howeve...
Key Factors:
- Incremental processing: The company wants to process only the daily incremental data, not the entire dataset every time.
- Low coding effort: The solution needs to minimize the need for custom development and configuration.
- Seamless integration with Glue: The solution should integrate directly with AWS Glue and its existing setup.
Option Analysis:
A) Create an ETL job that reads the S3 file status and logs the status in Amazon DynamoDB.
- Pros:
- This approach can track file status and changes manually by logging information into DynamoDB.
- Cons:
- This method involves a high level of manual effort and custom development to implement file status tracking and logging.
- Requires coding to manage the process of logging status and reading it for incremental processing.
- Adding DynamoDB introduces complexity and potential performance overhead.
- Use Case: While this can meet the requirement, it involves significant custom work, making it a more complex and error-prone solution compared to others.
B) Enable job bookmarks for the ETL jobs to update the state after a run to keep track of previously processed data.
- Pros:
- Job bookmarks are designed by AWS Glue specifically to handle incremental data processing.
- It enables the ETL jobs to track the state of data and only process new or updated data based on a timestamp or other defined key.
- Minimal coding effort: Enabling job bookmarks is straightforward and does not require significant code changes.
- Direct integration with AWS Glue and no additional services are required.
- Cons:
- It may require small changes in the job configuration, but the overall implementation is relatively simple.
- Use Case: This option is perfect for scenarios where you want incremental data processing with minimal coding and the existing Glue ...
Author: Joseph · Last updated Jul 30, 2026
An online retail company has an application that runs on Amazon EC2 instances that are in a VPC. The company wants to collect flow logs for the VPC and analyze network t...
Key Factors:
- Cost-effectiveness: The solution needs to minimize costs while still meeting the requirements of collecting and analyzing VPC flow logs.
- Ease of use: The solution should be simple to configure and maintain.
- Scalability and performance: The solution should be able to scale to handle large amounts of flow log data efficiently.
Option Analysis:
A) Publish flow logs to Amazon CloudWatch Logs. Use Amazon Athena for analytics.
- Pros:
- CloudWatch Logs is a managed service for log storage and analysis.
- Athena allows querying data stored in CloudWatch Logs.
- Cons:
- Storing flow logs in CloudWatch Logs can get expensive, especially with large volumes of log data.
- CloudWatch Logs are not designed for high-volume, cost-efficient log storage over the long term, which may make this solution less cost-effective.
- Athena querying over CloudWatch Logs might be less optimized for large datasets compared to S3-based solutions.
- Use Case: This could work for small-scale environments or scenarios where you need real-time log monitoring, but it may not be cost-efficient for large volumes of VPC flow logs.
B) Publish flow logs to Amazon CloudWatch Logs. Use an Amazon OpenSearch Service cluster for analytics.
- Pros:
- OpenSearch (formerly Elasticsearch) is a powerful search and analytics engine that integrates with CloudWatch Logs.
- Cons:
- OpenSearch can be costly for large datasets, especially in terms of storage and compute for processing flow logs.
- Similar to CloudWatch Logs, using OpenSearch for this type of data can quickly become expensive when dealing with large log volumes, especially if you're running it continuously for flow log analysis.
- Use Case: This solution is suitable for environments that require advanced search capabilities, but it's not the most cost-effective for analyzing flow logs in large volumes.
C) Publish flow logs to Amazon S3 i...
Author: Noah · Last updated Jul 30, 2026
A retail company stores transactions, store locations, and customer information tables in four reserved ra3.4xlarge Amazon Redshift cluster nodes. All three tables use even table distribution.The company updates the store location table only once or twice every few years.A data engineer notices that Redshift queues are slowing down because the whole store location table is constantly being broadcast to all four compute nodes for most queries. ...
Key Factors:
- Query performance: The goal is to speed up query performance by reducing the broadcasting of the store location table.
- Cost-effectiveness: The solution should optimize performance without increasing costs unnecessarily.
- Distribution style: The solution must address how data is distributed across the compute nodes to minimize unnecessary broadcasting.
Option Analysis:
A) Change the distribution style of the store location table from EVEN distribution to ALL distribution.
- Pros:
- ALL distribution broadcasts a copy of the table to each node, which can eliminate the need for shuffling data across nodes during joins. This is particularly useful for small, infrequently updated lookup tables like the store location table.
- Since the store location table is updated infrequently, broadcasting the table to all nodes is cost-effective and can significantly improve query performance.
- Cons:
- This may not be as effective if the table is large, but in this case, the table is infrequently updated, and changing to ALL distribution is a good fit.
- Use Case: This is the most cost-effective and performance-improving solution for this situation. It eliminates the need for broadcasting the store location table for every query while ensuring that other tables using EVEN distribution aren’t affected.
B) Change the distribution style of the store location table to KEY distribution based on the column that has the highest dimension.
- Pros:
- KEY distribution ensures that rows with the same value in the distribution column are stored on the same node, which is useful for queries that filter or join on the key column.
- Cons:
- While KEY distribution can reduce data movement in queries that filter or join based on the distribution key, it still does not guarantee that the store location table will not be broadcasted in certain cases.
- Choosing the right column for KEY distribution requires careful consideration of the data, and the store location table is unlikely to have a column with a high cardinality that would be ideal for this distribution method.
- If the store location table is small and updated infrequently, ALL distribution...
Author: CrystalWolfX · Last updated Jul 30, 2026
A company has a data warehouse that contains a table that is named Sales. The company stores the table in Amazon Redshift. The table includes a column that is named city_name. The company wants to query the table to find all row...
Key Factors:
- Requirement: The query must find all rows where the `city_name` starts with "San" or "El".
- SQL pattern matching: We are looking for a query that uses a regular expression to match city names starting with either "San" or "El".
Option Analysis:
A) Select from Sales where city_name ~ =‘$(San|El)’;
- Pros:
- The use of `~` is a valid regular expression match operator in Amazon Redshift.
- The pattern `$(San|El)` is not a correct regular expression syntax, and the `$(...)` part is incorrect in this context. The correct regular expression for "starts with" should be anchored to the start of the string using `^`.
- Cons:
- The regular expression is incorrect. The pattern doesn't properly capture the requirement to match city names starting with "San" or "El".
- Use Case: This is not a valid option due to incorrect syntax.
B) Select from Sales where city_name ~ =‘^(San|El)’;
- Pros:
- The `^` correctly anchors the match to the start of the string.
- The pattern `(San|El)` is correct for matching either "San" or "El" at the beginning of the string.
- Cons:
- The `` at the end of the regular expression is unnecessary. The `` means "zero or more occurrences" of the preceding element, but we only want to match the beginning of the string with either "San" or "El". The correct pattern should end with the group `(San|El)` without the ``.
- Use Case: This option is close but...
Author: Harper · Last updated Jul 30, 2026
A company needs to send customer call data from its on-premises PostgreSQL database to AWS to generate near real-time insights. The solution must capture and load updates from operational data stores that run in the PostgreSQL database. The data changes continuously.A data engineer configures an AWS Database Migration Service (AWS DMS) ongoing replication task. The task reads changes in near real time from the PostgreSQL source database transaction logs for each table. The task then sends the data to an Amazon Redshift cluster for processing.The data engineer discover...
To determine if the PostgreSQL database is the source of high latency, let's evaluate each option:
A) Use Amazon CloudWatch to monitor the DMS task. Examine the CDCIncomingChanges metric to identify delays in the CDC from the source database.
- This metric would show incoming changes that DMS is capturing, but it focuses on changes arriving in DMS, not specifically whether the source PostgreSQL database is causing the delay. This metric tracks changes entering DMS but doesn't directly confirm if the database itself is slow at producing these changes.
B) Verify that logical replication of the source database is configured in the postgresql.conf configuration file.
- This option checks if the PostgreSQL database is properly configured to support logical replication, which is needed for change data capture (CDC) by DMS. If the replication configuration is incorrect, it could result in delays. However, this step doesn't directly confirm if the database is slow at replicating or if the issue lies with DMS itself, making it less direct in isolating the source of latency.
C) Enable Amazon CloudWatch Logs for the DMS endpoin...
Author: Ethan · Last updated Jul 30, 2026
A lab uses IoT sensors to monitor humidity, temperature, and pressure for a project. The sensors send 100 KB of data every 10 seconds. A downstream process will read the data from an Amazon S3 bucket every...
Let's evaluate each solution to determine the best option for delivering IoT sensor data to an S3 bucket with the least latency:
Option A: Use Amazon Kinesis Data Streams and Amazon Kinesis Data Firehose to deliver the data to the S3 bucket. Use the default buffer interval for Kinesis Data Firehose.
- Why it’s rejected: Kinesis Data Firehose automatically batches data before delivering it to the destination (such as S3), and the default buffer interval is 300 seconds (5 minutes). This interval is much too long for the required 30-second data retrieval from the S3 bucket. The latency would be higher than required, so this option is not ideal for low-latency requirements.
Option B: Use Amazon Kinesis Data Streams to deliver the data to the S3 bucket. Configure the stream to use 5 provisioned shards.
- Why it’s rejected: Kinesis Data Streams is suitable for real-time data streaming, but delivering data to S3 requires a secondary step, such as using a custom process or Kinesis Data Firehose. Configuring 5 provisioned shards is a good way to scale, but it still involves extra steps to write the data to S3, leading to potential additional latency. The process will not be as streamlined and efficient as using Kinesis Data Firehose or other integrated solutions designed for low-latency delivery.
Option C: Use Amazon Kinesis Data Streams and call the Kinesis Client Library to deliver the data to the S3 bucket. Use a 5 second buffer interval from an application.
- Why it’s rejected: While Kinesis Data Streams with the Kinesis Client Library (KCL) can handle real-time data processing, implementing a custom application that pulls from the stream and writes to S3 introduces complexity and delays. Even though a 5-second buffer interval might reduce latency, the added overhead of managing the custom application and the manual writing of data to S3 makes this option less efficient and introduces operational complexity.
Option D:...
Author: Elizabeth · Last updated Jul 30, 2026
A company wants to use machine learning (ML) to perform analytics on data that is in an Amazon S3 data lake. The company has two data transformation requirements that will give consumers within the company the ability to create reports.The company must perform daily transformations on 300 GB of data that is in a variety format that must arrive in Amazon S3 at a scheduled time. The company must perform one-time transformations of terabytes of archived data that is in the S3 data lake. The company uses Amazon Managed Workflows for...
Let's evaluate the options based on the company’s needs for performing data transformations on daily and archived data in the most cost-effective manner.
A) For daily incoming data, use AWS Glue crawlers to scan and identify the schema.
- AWS Glue Crawlers are designed to automatically discover schema and metadata of data stored in Amazon S3. This solution is useful for identifying schema on daily incoming data, especially if the data is in varied formats (like CSV, JSON, Parquet, etc.). Once the schema is identified, Glue can perform data transformation tasks. This is a cost-effective solution for schema discovery, as Glue is serverless and can scale based on the data size. It’s suitable for periodic operations (like daily transformations) and minimizes the need for infrastructure management.
B) For daily incoming data, use Amazon Athena to scan and identify the schema.
- Amazon Athena is a serverless query service that allows you to analyze data directly in Amazon S3 using standard SQL. Athena can also identify the schema if the data is in formats like Parquet or ORC. However, Athena is more suitable for querying data rather than performing complex transformations, especially on a daily basis with large datasets. Athena charges based on the amount of data scanned, which can become costly for large datasets (e.g., 300 GB daily) when performing frequent scans. While useful for queries, Athena is less cost-effective than AWS Glue for daily data transformations and schema management.
C) For daily incoming data, use Amazon Redshift to perform transformations.
- Amazon Redshift is a data warehousing service optimized for large-scale data analysis. While it can perform data transformations, it is more suited for complex analytics and queries on structured data. Redshift incurs higher costs due to provisioning compute resources, making it less cost-effective for daily data transformations when compared to serverless options like...
Author: Charlotte · Last updated Jul 30, 2026
A retail company uses AWS Glue for extract, transform, and load (ETL) operations on a dataset that contains information about customer orders. The company wants to implement specific validation rule...
Let's evaluate each option based on the need to implement specific validation rules to ensure data accuracy and consistency.
A) Use AWS Glue job bookmarks to track the data for accuracy and consistency.
- AWS Glue job bookmarks are used to keep track of processed data between ETL job runs, ensuring that only new or modified data is processed in subsequent jobs. While job bookmarks are useful for incremental data processing, they are not specifically designed for validation or enforcing data accuracy and consistency. Job bookmarks track progress but do not allow you to define or check validation rules. Therefore, this option does not meet the need for specific data validation rules.
B) Create custom AWS Glue Data Quality rulesets to define specific data quality checks.
- AWS Glue Data Quality allows you to create custom rulesets to define specific data quality checks based on your needs (e.g., range checks, null value checks, uniqueness checks, etc.). This solution is ideal for implementing specific validation rules to ensure data accuracy and consistency. You can define rules and apply them to your data in a flexible and granular way. This makes it the most suitable solution for the company's requirement to implement custom validation rules.
C) Use the built-in AWS Glue Data Quality transforms for standard data quality validations.
- AWS Glue Data Quality transforms provide pre-built data quality checks such ...
Author: VioletCheetah55 · Last updated Jul 30, 2026
An insurance company stores transaction data that the company compressed with gzip.The company needs to query the transaction data for occasional audits.Which...
Let's analyze each option to find the most cost-effective solution for the insurance company to query the transaction data for occasional audits:
A) Store the data in Amazon Glacier Flexible Retrieval. Use Amazon S3 Glacier Select to query the data.
- Amazon Glacier Flexible Retrieval (previously known as Amazon Glacier) is an archival storage solution optimized for infrequently accessed data with low retrieval costs. However, Glacier is designed for long-term archival and does not support fast query performance. Querying data with S3 Glacier Select can be slow and incurs additional costs for retrieval, which may not be the most cost-effective solution for occasional audits. Glacier is best suited for data that is rarely accessed and needs to be retrieved infrequently, making this option less suitable for the company's needs.
B) Store the data in Amazon S3. Use Amazon S3 Select to query the data.
- Amazon S3 Select enables querying compressed data directly within Amazon S3 without needing to retrieve the entire object. This is a very cost-effective solution when dealing with small to medium-sized datasets stored in S3, especially if the data is already compressed (gzip) in a compatible format (e.g., CSV, JSON, or Parquet). S3 Select can be used to quickly retrieve specific data subsets for occasional audits, making it an efficient and low-cost option for querying compressed data.
C) Store the data in Amazon S3. Use Amazon Athena to query the data.
- Amazon Athena is a serverless query service that allows SQL queries on data stored in Amazon S3, including support for compressed files like gzip. While Athena provides powerful querying capabilities, it charges based on the am...
Author: Aria · Last updated Jul 30, 2026
A data engineer finished testing an Amazon Redshift stored procedure that processes and inserts data into a table that is not mission critical. The engineer wants to automatically run the stored procedure o...
To meet the requirement of running the stored procedure daily in a cost-effective manner, we need to consider the various options based on factors like cost, ease of use, and scalability.
Option A: Create an AWS Lambda function to schedule a cron job to run the stored procedure.
- Pros: AWS Lambda is serverless, meaning no infrastructure to manage, and you only pay for the actual execution time, making it a cost-effective option for lightweight tasks. It can be easily set up with AWS CloudWatch Events (EventBridge) to schedule the procedure execution.
- Cons: Lambda has a maximum execution time limit (15 minutes), so if the stored procedure runs longer than that, it might not work. In addition, Lambda might not be the best solution if there are multiple steps or complex workflows required.
- Best for: Simple tasks that require minimal execution time and low cost.
Option B: Schedule and run the stored procedure by using the Amazon Redshift Data API in an Amazon EC2 Spot Instance.
- Pros: EC2 Spot Instances offer significant cost savings when compared to on-demand instances.
- Cons: Using EC2 requires you to manage the instance lifecycle, even if it's a Spot instance. Spot instances can be terminated at any time, potentially leading to interruptions in the execution of the stored procedure. Additionally, the Data API itself is a bit more complex and may ...
Author: Oliver · Last updated Jul 30, 2026
A marketing company collects clickstream data. The company sends the clickstream data to Amazon Kinesis Data Firehose and stores the clickstream data in Amazon S3. The company wants to build a series of dashboards that hundreds of users from multiple departments will use.The company will use Amazon QuickSight to develop the dashboards. The company wants a solution t...
To meet the requirement of creating a series of dashboards for hundreds of users with daily updates about clickstream data, let's evaluate each option carefully.
A) Use Amazon Redshift to store and query the clickstream data.
- Pros: Amazon Redshift is a powerful, scalable data warehouse that can handle large amounts of data and complex queries. It integrates well with Amazon QuickSight for dashboarding.
- Cons: While Redshift is highly capable, it is typically more expensive than other solutions like Amazon Athena or S3 for this use case, particularly if the clickstream data is stored in Amazon S3. You would need to load data into Redshift and pay for compute and storage costs, which could become costly over time.
- Use case: Redshift is ideal for large-scale analytics workloads with complex joins and aggregations, but it may not be the most cost-effective option for simple querying of clickstream data stored in S3, especially when the data updates daily.
B) Use Amazon Athena to query the clickstream data.
- Pros: Amazon Athena allows you to directly query data stored in Amazon S3 using SQL, without needing to load the data into a separate service like Redshift. Athena is serverless, meaning you only pay for the queries you run, which can be very cost-effective for this type of use case. It’s a good fit for querying large amounts of raw clickstream data in S3.
- Cons: Athena’s performance can vary depending on how the data is structured in S3. However, for simple query workloads and daily updates, Athena’s cost structure is advantageous.
- Use case: Athena is ideal for ad-hoc querying and scalable, cost-effective querying of large datasets stored in S3, especially if the queries are relatively straightforward, as in this case.
C) Use Amazon S3 analytics to query the clickstream data.
- Pros: S3 analytics helps optimize queries and manage data lifecycle policies, but it is not specifically designed for querying data. It's mainly used for tracking data access patterns to optimize storage, not for querying large amounts of data.
- Cons: S3 analytics does not allow for querying or aggregating clickstream data in the way Athena or Redshift can. It's a tool designed to optimize data storage, not for querying or ...
Author: Isabella1 · Last updated Jul 30, 2026
A data engineer is building a data orchestration workflow. The data engineer plans to use a hybrid model that includes some on-premises resources and some resources that are in the cloud. The data engineer wants to prioritize portability and open source resource...
To address the need for a hybrid data orchestration workflow that prioritizes portability and open-source resources, let’s evaluate the available options:
A) AWS Data Exchange
- Pros: AWS Data Exchange enables the sharing of third-party data, but it's primarily focused on data sharing and not orchestration. It helps in finding, subscribing to, and using data from different providers, rather than orchestrating workflows or processes.
- Cons: AWS Data Exchange does not provide capabilities for orchestrating workflows or managing processes in both on-premises and cloud environments. It is not designed for the hybrid orchestration scenario.
- Use case: This service would be suitable if the goal were to subscribe to external data sources, but it does not address the requirements for orchestration.
B) Amazon Simple Workflow Service (Amazon SWF)
- Pros: Amazon SWF is a fully managed service that helps developers coordinate tasks in a distributed application. It can handle workflows that span both on-premises and cloud-based resources.
- Cons: SWF is a proprietary AWS service and is not based on open-source frameworks. This means it lacks the portability that the data engineer is looking for, especially for hybrid environments where a flexible, open-source solution would be preferred.
- Use case: SWF could be used for workflows, but its lack of open-source nature makes it less ideal in this case.
C) Amazon Managed Workflows for Apache Airflow (Amazon MWAA)
- Pros: Apache Airflow is an open-source workflow management platform that is widely used for orchestrating complex workflows. Amazon MWAA is a managed service that provid...
Author: Isabella · Last updated Jul 30, 2026
A gaming company uses a NoSQL database to store customer information. The company is planning to migrate to AWS.The company needs a fully managed AWS solution that will handle high online transaction processing (OLTP) workload, provide single-digit millisecond performance, and pr...
To meet the requirements of handling high online transaction processing (OLTP) workloads, providing single-digit millisecond performance, and ensuring high availability around the world, let's evaluate each option:
A) Amazon Keyspaces (for Apache Cassandra)
- Pros: Amazon Keyspaces is a managed service for Apache Cassandra, a highly scalable NoSQL database. It is ideal for large-scale, high-throughput, and low-latency workloads. It offers good performance and scalability.
- Cons: While Keyspaces can handle large-scale workloads, it may require more operational management compared to some of the other AWS services. Cassandra, even in a managed environment like Keyspaces, often requires additional tuning and expertise to optimize performance and maintain availability across global regions.
- Use case: Keyspaces is a good choice for large-scale distributed databases that require extensive customization and tuning but might not offer the easiest operational overhead compared to other options on AWS.
B) Amazon DocumentDB (with MongoDB compatibility)
- Pros: Amazon DocumentDB is a fully managed database service that is compatible with MongoDB. It is designed to handle high workloads and provide scalability for applications that require document-oriented storage.
- Cons: DocumentDB is typically optimized for document-based data and might not be the best fit for high-throughput, low-latency OLTP workloads. While it provides good scalability and availability, it doesn’t always provide the same single-digit millisecond response times as DynamoDB, especially for high-velocity transactional workloads.
- Use case: DocumentDB is more suitable for applications that require MongoDB compatibility and document-based storage. It can handle substantial workloads, but may not provide the best performance for high-throughput OLTP applications.
C) Amazon Dy...
Author: Sophia · Last updated Jul 30, 2026
A data engineer creates an AWS Lambda function that an Amazon EventBridge event will invoke. When the data engineer tries to invoke the Lambda function by using an EventBridge event, an Access...
When the data engineer encounters an AccessDeniedException while invoking an AWS Lambda function using an Amazon EventBridge event, the issue likely arises from permission misconfigurations either with the EventBridge event or the Lambda function. Let’s evaluate each option:
A) Ensure that the trust policy of the Lambda function execution role allows EventBridge to assume the execution role.
- Pros: This option focuses on the trust relationship, where EventBridge must have permission to invoke the Lambda function. However, for an EventBridge rule to invoke Lambda, it needs to have proper permissions in both the EventBridge rule's role and the Lambda function's resource policy, not just the Lambda execution role trust policy.
- Cons: The trust policy mainly governs what services (like EventBridge) can assume the Lambda execution role. However, the problem is typically more about what EventBridge is allowed to do (invoke the Lambda) rather than the role assumption itself.
- Use case: This option would only apply if there is an issue with Lambda execution role assumption, which is not the root cause of the AccessDeniedException in this scenario.
B) Ensure that both the IAM role that EventBridge uses and the Lambda function's resource-based policy have the necessary permissions.
- Pros: This is the most likely solution. EventBridge needs the correct permissions to invoke the Lambda function, and Lambda needs to allow EventBridge to invoke it. This can be achieved by configuring the Lambda function's resource-based policy to grant EventBridge permission. Additionally, EventBridge needs the right IAM role permissions to trigger the Lambda function.
- Cons: This is the correct solution, and addressin...
Author: Ishaan · Last updated Jul 30, 2026
A company uses a data lake that is based on an Amazon S3 bucket. To comply with regulations, the company must apply two layers of server-side encryption to files that are uploaded to the S3 bucket. The company wants to use an ...
To determine the correct solution for the company's requirement to apply two layers of server-side encryption (SSE) to files uploaded to the Amazon S3 bucket, let's analyze the options based on key factors such as compliance, security, and operational ease. The company wants to use an AWS Lambda function to apply the encryption.
A) Use both server-side encryption with AWS KMS keys (SSE-KMS) and the Amazon S3 Encryption Client.
- Reasoning: This option proposes using two encryption layers: SSE-KMS (for encrypting data using AWS Key Management Service) and the Amazon S3 Encryption Client (likely implying client-side encryption before upload).
- Why rejected: While this approach could work in theory, it requires significant manual management of the encryption process, including configuring the S3 Encryption Client in the Lambda function, which can be cumbersome. The S3 Encryption Client is client-side encryption, so it would apply encryption on the client side before the data reaches S3, thus complicating the process without providing an ideal method for automated, server-side encryption after the upload.
B) Use dual-layer server-side encryption with AWS KMS keys (DSSE-KMS).
- Reasoning: This solution offers two layers of server-side encryption using KMS keys (DSSE-KMS). It implies the use of two distinct encryption mechanisms on the server side after the file reaches S3.
- Why rejected: AWS does not natively offer "dual-layer" encryption as a named service. While SSE-KMS allows you to apply server-side encryption with KMS, there is no such option as DSSE-KMS for automatic dual-layer encryption. A Lambda fun...
Author: Olivia Johnson · Last updated Jul 30, 2026
A data engineer notices that Amazon Athena queries are held in a queue before the queries run.How can the d...
Let's analyze each option in detail to determine the best way to prevent queries from being queued in Amazon Athena:
A) Increase the query result limit
- Explanation: The query result limit determines the amount of data that can be returned by a query. While this setting controls the size of query results, it has no effect on how queries are queued or how resources are allocated for query execution.
- Rejection Reason: Increasing the query result limit doesn't prevent queries from being queued. It only affects the size of the result set returned. This does not solve the root issue of query queuing.
B) Configure provisioned capacity for an existing workgroup
- Explanation: Amazon Athena allows you to configure a workgroup with provisioned capacity, which ensures that a certain amount of resources (e.g., CPU, memory) are allocated to a specific workgroup. With provisioned capacity, Athena queries in that workgroup will have dedicated resources and avoid being queued, as the capacity is pre-allocated.
- Why this works: Provisioned capacity directly addresses the issue of queries being queued because it guarantees that the required resources are available for query execution without waiting for available capacity.
- Rejection Reason: None, this option directly solves the problem of query queuing.
C) Use federated queries
...
Author: CrimsonViperX · Last updated Jul 30, 2026
A data engineer needs to debug an AWS Glue job that reads from Amazon S3 and writes to Amazon Redshift. The data engineer enabled the bookmark feature for the AWS Glue job.The data engineer has set the maximum concurrency for the AWS Glue job to 1.The AWS Glue job is successfully writing the output to Amazon Redshift. However, the Amazon S3 files that were lo...
To understand why the AWS Glue job is reprocessing the files from Amazon S3 that were already loaded in previous runs, we need to explore the behavior of AWS Glue's bookmark feature and how it interacts with job settings and permissions.
A) The AWS Glue job does not have the `s3:GetObjectAcl` permission that is required for bookmarks to work correctly.
- Reasoning: The bookmark feature in AWS Glue uses metadata to track which files have already been processed to avoid reprocessing. This metadata is stored in a specific location (usually in the Glue Data Catalog) and relies on the AWS Glue job having sufficient permissions to track the objects' status in S3.
- Why rejected: While it’s true that the `s3:GetObjectAcl` permission is necessary for certain Glue functionalities, this specific permission isn't directly related to the bookmark feature. The missing permission would more likely cause a different issue (like failing to access or read S3 objects) rather than the reprocessing of already-loaded files. The problem here seems to be related to how the bookmarks are being tracked rather than missing permissions.
B) The maximum concurrency for the AWS Glue job is set to 1.
- Reasoning: The maximum concurrency setting controls the number of concurrent tasks that AWS Glue can run for a job. Setting this to 1 ensures that only one task runs at a time, but this is not directly related to the bookmark feature or why files are being reprocessed.
- Why rejected: While setting the concurrency to 1 can affect the speed and parallelism of the job, it does not explain why the bookmark feature isn't preventing reprocessing. The problem appears to be related to the state or tracking of processed files, not the concurrency of the job itself.
C) The...
Author: Vikram · Last updated Jul 30, 2026
An ecommerce company wants to use AWS to migrate data pipelines from an on-premises environment into the AWS Cloud. The company currently uses a third-party tool in the on-premises environment to orchestrate data ingestion processes.The company wants a migration solution that does not require the company to manage servers. The solution must be able to orchestrate Python ...
To identify the best solution for migrating data pipelines from an on-premises environment into the AWS Cloud, we need to focus on the requirements:
- No server management (serverless solution).
- Orchestration of Python and Bash scripts.
- Minimal or no code refactoring.
Let’s evaluate the options based on these key factors:
A) AWS Lambda
- Reasoning: AWS Lambda is a serverless compute service that allows you to run code without provisioning or managing servers. It supports Python and Bash scripts via Lambda functions. However, Lambda is typically suited for short-duration tasks (maximum runtime of 15 minutes) and can be triggered by events, but it is not designed as an orchestration tool for complex workflows or managing multiple steps in a pipeline.
- Why rejected: While AWS Lambda could technically execute individual Python or Bash scripts, it is not an ideal solution for orchestrating a series of scripts. Lambda requires each step to be triggered and managed separately, leading to potentially complex configurations. It's not designed for managing or chaining together multi-step workflows.
B) Amazon Managed Workflows for Apache Airflow (Amazon MWAA)
- Reasoning: Apache Airflow is a popular open-source tool for orchestrating workflows, and Amazon MWAA provides a managed service for running Apache Airflow in AWS. Airflow is highly flexible, allowing the orchestration of complex data pipelines. It can easily handle Python and Bash scripts.
- Why rejected: While Amazon MWAA is a powerful tool for orchestrating complex workflows, it requires more operational management compared to other solutions (e.g., setting up the environment, configuring DAGs, managing Airflow resources). The operational overhead can be higher than other options, and since the company wants to avoid managing servers, this solution may not be the best fit for the “least operation...
Author: Evelyn · Last updated Jul 30, 2026
A retail company stores data from a product lifecycle management (PLM) application in an on-premises MySQL database. The PLM application frequently updates the database when transactions occur.The company wants to gather insights from the PLM application in near real time. The company wants to integrate the insights with other business datasets and to analyze the combined dataset by using an Amazon Redshift data warehouse.The comp...
Let's analyze each option to determine the best solution for integrating the PLM application’s MySQL database updates into Amazon Redshift with the least development effort.
A) Run a scheduled AWS Glue extract, transform, and load (ETL) job to get the MySQL database updates by using a Java Database Connectivity (JDBC) connection. Set Amazon Redshift as the destination for the ETL job.
- Explanation: AWS Glue is a fully managed ETL service, and it can connect to MySQL databases via JDBC to extract data. You can configure the ETL job to run on a schedule to pull data from MySQL and load it into Amazon Redshift. However, this option involves scheduling jobs to run at intervals, which means it may not achieve near real-time updates. The scheduled nature of the job introduces latency between updates in the MySQL database and when the data is available in Redshift.
- Rejection Reason: While AWS Glue is a good solution for ETL, it introduces some latency due to its scheduled nature and is not optimal for near real-time integration.
B) Run a full load plus CDC task in AWS Database Migration Service (AWS DMS) to continuously replicate the MySQL database changes. Set Amazon Redshift as the destination for the task.
- Explanation: AWS DMS (Database Migration Service) supports change data capture (CDC) to replicate data from MySQL to Amazon Redshift continuously in near real time. DMS can perform full loads followed by incremental replication (CDC), which means it will continuously capture and migrate changes from the MySQL database to Redshift, offering near real-time updates.
- Why this works: AWS DMS is a fully managed service designed for database replication with minimal setup. It provides an easy way to replicate MySQL database changes to Redshift continuously, achieving the real-time integration that the company needs. DMS also offers robust support for ongoing database updates and replication with minimal development effort.
- Rejection Reason: None. This solution provides the requ...
Author: Victoria · Last updated Jul 30, 2026
A marketing company uses Amazon S3 to store clickstream data. The company queries the data at the end of each day by using a SQL JOIN clause on S3 objects that are stored in separate buckets.The company creates key performance indicators (KPIs) based on the objects. The company needs a serverless solution that will give users the ability to query data by partitioning the data. The solut...
In this scenario, the company needs a serverless solution for querying the clickstream data stored in Amazon S3, with ACID properties for transaction management, the ability to partition the data, and cost-effectiveness.
Evaluating each option:
A) Amazon S3 Select:
- Pros: Amazon S3 Select allows you to query specific data within an S3 object, reducing the amount of data transferred.
- Cons: While this option is serverless and cost-effective for reading small portions of data, S3 Select does not provide ACID guarantees nor can it perform complex queries like JOIN operations or handle large datasets efficiently. It also does not offer partitioning or indexing for optimized query performance at scale.
B) Amazon Redshift Spectrum:
- Pros: Amazon Redshift Spectrum allows querying data directly from S3, and it integrates well with Amazon Redshift. It supports complex SQL queries with joins, aggregations, and partitioning. It also provides ACID guarantees for transactions.
- Cons: Redshift Spectrum is not serverless because it requires a running Redshift cluster, which incurs additional costs for cluster uptime and maintenance. This increases the overall cost compared to fully serverless options.
C) Amazon Athena:
- Pros: Amazon Athena is a fully serverless service that allows you to query S3 data using SQL. It can handle large datasets, supports partitioning, and allows querying with SQL JOIN operations. It also provides ACID compliance when used in conjunction with AWS Glue and partitioning strategies. Athena is highly cost-effective...
Author: Aarav · Last updated Jul 30, 2026
A company wants to migrate data from an Amazon RDS for PostgreSQL DB instance in the eu-east-1 Region of an AWS account named Account_A. The company will migrate the data to an Amazon Redshift cluster in the eu-west-1 Region of an AWS account named Account_B.Which ...
To migrate data using AWS Database Migration Service (AWS DMS) between two AWS accounts and regions (from Account_A in eu-east-1 to Account_B in eu-west-1), we need to ensure the following:
1. Replication instance location: The replication instance must be able to access both the source database (RDS for PostgreSQL in eu-east-1) and the target database (Amazon Redshift in eu-west-1). This means the replication instance needs to be in a region that is reachable by both source and target.
2. Cross-account access: DMS replication instances need to have the correct network and IAM permissions to interact with databases in different AWS accounts. This involves configuring network settings (VPC, subnets, security groups) and IAM roles for cross-account access.
Evaluating each option:
A) Set up an AWS DMS replication instance in Account_B in eu-west-1:
- Pros: This option places the replication instance in Account_B, in the same region as the target Redshift cluster (eu-west-1). It simplifies network access to the target (Amazon Redshift) because the replication instance is already within the same region as the target database. It can also have direct access to Account_B's resources.
- Cons: The replication instance will need cross-account access to the source database in Account_A (eu-east-1). This is achievable with the proper IAM role and network configuration (VPC peering, VPN, etc.). This setup is a common and recommended configuration.
B) Set up an AWS DMS replication instance in Account_B in eu-east-1:
- Cons: While the replication instance is in Account_B, it is in the wrong region (eu-east-1) and will create latency and potential network issues because the target Redshift cluster is located in eu-west-1. Cross-region rep...
Author: Zain · Last updated Jul 30, 2026
A company uses Amazon S3 as a data lake. The company sets up a data warehouse by using a multi-node Amazon Redshift cluster. The company organizes the data files in the data lake based on the data source of each data file.The company loads all the data files into one table in the Redshift cluster by using a separate COPY command for each data file location. This approach takes a long time to load all the data files i...
In this scenario, the company wants to increase the speed of data ingestion into an Amazon Redshift cluster while avoiding an increase in costs. The company is currently using separate `COPY` commands for each data file, which is inefficient. Let's evaluate each option based on the requirements:
A) Use a provisioned Amazon EMR cluster to copy all the data files into one folder. Use a COPY command to load the data into Amazon Redshift.
- Pros: Amazon EMR is a powerful tool for processing large volumes of data and can help in managing and preprocessing large datasets. If used correctly, it can handle parallel processing of files.
- Cons: Provisioned EMR clusters can be costly, especially for large datasets, which contradicts the requirement of not increasing the cost of the process. Also, the step of copying the data into one folder doesn’t directly address the problem of inefficient `COPY` commands or optimize the Redshift loading process.
B) Load all the data files in parallel into Amazon Aurora. Run an AWS Glue job to load the data into Amazon Redshift.
- Cons: This solution introduces Amazon Aurora as an intermediate storage, which complicates the workflow and increases both operational complexity and cost. Additionally, loading data into Amazon Aurora before copying to Redshift introduces unnecessary overhead.
C) Use an AWS Glue job to copy all the data files into one folder. Use a COPY command to load the data into Amazon Redshift.
- Cons: While AWS Glue can help with orchestrating the process of moving data between sources, copying data into one fold...
Author: CrimsonViperX · Last updated Jul 30, 2026
A company plans to use Amazon Kinesis Data Firehose to store data in Amazon S3. The source data consists of 2 MB .csv files. The company must convert the .csv files to JSON format. The company must store the files in Apac...
To meet the requirements of converting `.csv` files to `JSON` format and storing the resulting files in Apache Parquet format, we need to evaluate the solutions based on development effort, integration with Kinesis Data Firehose, and the least operational complexity.
Let’s examine the options:
A) Use Kinesis Data Firehose to convert the .csv files to JSON. Use an AWS Lambda function to store the files in Parquet format.
- Reasoning: Kinesis Data Firehose supports direct conversion from formats like JSON, CSV, and Apache ORC, but it cannot convert to Parquet on its own. The option involves using Firehose to convert the files to JSON and then using Lambda to store them in Parquet format.
- Why rejected: This option involves extra development effort because you would have to write and maintain a custom Lambda function to convert the data into Parquet format after the JSON conversion. This adds complexity to the solution and is less efficient than a more integrated approach.
B) Use Kinesis Data Firehose to convert the .csv files to JSON and to store the files in Parquet format.
- Reasoning: Kinesis Data Firehose has built-in support for converting `.csv` to `JSON` but does not have native support for converting to Parquet format directly.
- Why rejected: This option would not work because Kinesis Data Firehose cannot convert the `.csv` files to Parquet directly. While it can convert data to JSON, it requires an additional step for the conversion to Parquet, which isn't directly supported in this setup.
C) Use Kinesis Data Firehose to invoke an AWS Lambda function that transforms the .csv files to JSON and...
Author: Maya2022 · Last updated Jul 30, 2026
A company is using an AWS Transfer Family server to migrate data from an on-premises environment to AWS. Company policy mandates the use of TLS 1.2 or above to en...
To meet the company's policy of using TLS 1.2 or above for data encryption in transit with AWS Transfer Family, let's evaluate each option carefully:
A) Generate new SSH keys for the Transfer Family server. Make the old keys and the new keys available for use.
- Cons: SSH keys are used for authentication purposes, not for encrypting data in transit. SSH keys cannot enforce TLS 1.2 encryption; they are related to securing the connection and verifying the identity of the user or client. This approach does not address the encryption protocol requirement (TLS 1.2) specified by the policy.
B) Update the security group rules for the on-premises network to allow only connections that use TLS 1.2 or above.
- Cons: Security group rules are used to control access to the server by specifying allowed IP ranges, ports, and protocols, but they cannot enforce encryption standards. Security groups do not have any control over which version of TLS or other encryption protocols are used for communication. Therefore, this approach would not meet the TLS 1.2 encryption requirement.
C) Update the security policy of the Transfer Family server to specify a minimum protocol version of TLS 1.2.
- Pros: AWS Transfer Family supports the ability to configure the minimum protocol version for secure transfers. By specifying a minimum pr...
Author: SilverBear · Last updated Jul 30, 2026
A company wants to migrate an application and an on-premises Apache Kafka server to AWS. The application processes incremental updates that an on-premises Oracle database sends to the Kafka server. The company wants to use the replatform migration strategy i...
To migrate an on-premises Apache Kafka server to AWS with the least management overhead using a replatform strategy, we need to consider key factors such as ease of management, compatibility with the existing application, and scalability requirements.
Option A: Amazon Kinesis Data Streams
- Rejected: Amazon Kinesis Data Streams is a fully managed service for streaming data, but it is not directly compatible with Apache Kafka. The company is using Apache Kafka, which means a transition to Kinesis Data Streams would require substantial changes to the application and its architecture, which contradicts the replatform strategy (which aims for minimal changes).
Option B: Amazon Managed Streaming for Apache Kafka (Amazon MSK) provisioned cluster
- Rejected: While Amazon MSK provisioned clusters offer a fully managed Apache Kafka service, they still require some level of management (e.g., capacity planning, monitoring, and scaling). This option adds more management overhead compared to the serverless option, which is what the company is trying to avoid for a simpler migration.
Option C: Amazon Kinesis Data Firehose
- Rejected: Amazon Kinesis Data Firehose is a fully managed service for delivering streaming data to destinations like Amazon S3, Amazon Redshift, or Amazon Elasticsearch. However, it is not directly compatible with Kafka and would also ...
Author: NightmareDragon2025 · Last updated Jul 30, 2026
A data engineer is building an automated extract, transform, and load (ETL) ingestion pipeline by using AWS Glue. The pipeline ingests compressed files that are in an Amazon S3 bucket. The ingestion pipeline must support increme...
To meet the requirement of building an automated ETL pipeline that supports incremental data processing, the data engineer needs a feature that helps track which data has already been processed. This is especially important for incremental processing to ensure that only new or updated data is processed.
Option A: Workflows
- Rejected: AWS Glue Workflows are designed to manage and orchestrate a series of jobs and crawlers. While workflows are useful for managing complex ETL pipelines with multiple jobs and dependencies, they do not directly handle incremental processing or data tracking. Thus, workflows are not the right choice for supporting incremental data ingestion.
Option B: Triggers
- Rejected: AWS Glue Triggers are used to schedule and initiate jobs based on specific events or time-based schedules. While triggers can automate the ETL process, they do not inherently handle incremental data processing or track which data has already been processed. They simply initiate the job execution, which means additional logic would be required to handle the incremental aspect.
Option C: Job bookmarks
- Selected: AWS Glue Job Bookmarks are specifically designed for incremental data processing. When enabled, job bookmarks keep track of the state of the data that has a...
Author: SolarFalcon11 · Last updated Jul 30, 2026
A banking company uses an application to collect large volumes of transactional data. The company uses Amazon Kinesis Data Streams for real-time analytics. The company=E2=80=99s application uses the PutRecord action to send data to Kinesis Data Streams.A data engineer has observed network outages during certain times of da...
To configure exactly-once delivery for the entire processing pipeline, we need to ensure that the data is processed once and only once, even in the case of network outages or other interruptions. Let’s evaluate the given options based on their ability to guarantee exactly-once processing while taking into account factors like reliability, ease of implementation, and potential overhead.
A) Design the application so it can remove duplicates during processing by embedding a unique ID in each record at the source.
- Reasoning: By embedding a unique ID in each record, the application could check for duplicate records at the processing stage, and filter them out. This would prevent duplicates during the processing step.
- Why rejected: While this approach can handle duplicate data during processing, it doesn’t prevent duplicates from being sent to Kinesis Data Streams in the first place. Additionally, it places the burden of deduplication on the application logic, which can be error-prone and increase complexity. This solution does not guarantee exactly-once delivery, but only ensures deduplication during processing.
B) Update the checkpoint configuration of the Amazon Managed Service for Apache Flink (previously known as Amazon Kinesis Data Analytics) data collection application to avoid duplicate processing of events.
- Reasoning: Amazon Managed Service for Apache Flink supports exactly-once semantics through checkpointing, which ensures that processed records are stored periodically and that data is reprocessed from the last checkpoint in case of a failure. This can prevent duplicate processing of records in a real-time stream.
- Why selected: Flink's exactly-once semantics and its checkpointing mechanism are specifically designed to ensure reliable delivery, even in the event of network outages or failures. By updating the checkpoint configuration, the data engineer can configure Flink to process records exac...
Author: Rohan · Last updated Jul 30, 2026
A company stores logs in an Amazon S3 bucket. When a data engineer attempts to access several log files, the data engineer discovers that some files have been unintentionally deleted.The data engineer needs a solution that will prevent unintentional...
To prevent unintentional file deletion in Amazon S3 with the least operational overhead, the solution must provide a mechanism to retain deleted data and prevent accidental removal. Let's examine each option:
Option A: Manually back up the S3 bucket on a regular basis
- Rejected: Manually backing up the S3 bucket would require ongoing effort to regularly copy data to a separate location, which introduces significant operational overhead. Additionally, it does not prevent the original data from being deleted; it merely provides a copy elsewhere. This solution is inefficient and prone to human error.
Option B: Enable S3 Versioning for the S3 bucket
- Selected: Enabling Amazon S3 Versioning is the best solution to prevent unintentional file deletion with the least operational overhead. When versioning is enabled, S3 retains all versions of an object, including deleted versions. If a file is accidentally deleted, the previous version of the file can be restored easily. This solution provides a built-in, automatic mechanism for recovering from accidental deletions, with minimal management required after enabling it. It ensures that no data is truly lost and provides a simple recovery method without needing separate backup mechanisms.
Option C: Configure replication for the S3 bucket
- Rejected: While S3 replication could help create ...
Author: Lucas Carter · Last updated Jul 30, 2026
A telecommunications company collects network usage data throughout each day at a rate of several thousand data points each second. The company runs an application to process the usage data in real time. The company aggregates and stores the data in an Amazon Aurora DB instance.Sudden drops in network usage usually indicate a network outage. The company must be able...
To meet the requirement of detecting sudden drops in network usage with the least latency, we need to focus on real-time processing and minimize delays between the event (drop in network usage) and the action (remedial response). Let's evaluate each option:
Option A: Create an AWS Lambda function to query Aurora for drops in network usage. Use Amazon EventBridge to automatically invoke the Lambda function every minute.
- Rejected: Querying Aurora every minute using Lambda introduces unnecessary delay since it's not truly real-time. The "minute-based" frequency of invocation increases latency, which doesn't align with the company's need for near-instantaneous response to network usage drops. The polling interval is too slow, and this solution would not be efficient for detecting network usage drops that could happen suddenly.
Option B: Modify the processing application to publish the data to an Amazon Kinesis data stream. Create an Amazon Managed Service for Apache Flink (previously known as Amazon Kinesis Data Analytics) application to detect drops in network usage.
- Selected: This is the optimal solution. By using Kinesis Data Streams, the company can stream network usage data in real time. With Amazon Managed Service for Apache Flink, the company can apply real-time analytics to detect sudden drops in usage with low latency. Flink supports processing large amounts of streaming data and can immediately detect any anomalies (such as a sudden drop in network usage). This provides the least latency as it continuously processes data and can trigger remedial actions almost instantaneously.
Option C: Replace the Auror...
Author: Emma · Last updated Jul 30, 2026
A data engineer is processing and analyzing multiple terabytes of raw data that is in Amazon S3. The data engineer needs to clean and prepare the data. Then the data engineer needs to load the data into Amazon Redshift for analytics.The data engineer needs a solution that will give data analysts the ability to perform complex queries. The solution must eliminate the need to perfo...
To meet the requirements of cleaning and preparing large amounts of raw data and loading it into Amazon Redshift for analytics with minimal operational overhead, let’s evaluate the options:
Option A: Amazon EMR, AWS Step Functions, and Amazon QuickSight
- Amazon EMR: Amazon EMR is a powerful tool for processing large datasets using Apache Hadoop, Spark, or other frameworks. While it’s highly scalable, it requires setting up and managing the cluster, which adds operational complexity.
- AWS Step Functions: While Step Functions is great for orchestrating workflows, it doesn’t directly simplify the process of data preparation or transformation. It would still need an integration with tools like AWS Lambda or Amazon EMR, which leads to increased complexity.
- Amazon QuickSight: QuickSight is a BI tool, but it is not suitable for performing complex queries directly on raw or transformed data. It is better suited for visualization and dashboarding, not for querying large datasets in a Redshift data warehouse.
This option is rejected because it requires managing an EMR cluster and doesn’t provide a streamlined, low-maintenance ETL process.
Option B: AWS Glue DataBrew, AWS Glue, and Amazon Redshift
- AWS Glue DataBrew: AWS Glue DataBrew is a powerful no-code data preparation tool that enables data engineers to clean and transform data without writing code. It integrates well with S3 and is easy to use for data preparation.
- AWS Glue: AWS Glue can be used to load the prepared data into Amazon Redshift. It is a fully managed ETL service, meaning minimal infrastructure management and scalability are built-in. It is designed for easy integration with Redshift.
- Amazon Redshift: Redshift is a fully managed data warehouse, which is ideal for running complex queries on large datasets, making it a good fit for analytics.
This option is ideal because it eliminates the need to manage infrastructure, simplifies the ETL process, and makes it easy to run complex queries on prepared data. It also leverages fully managed services, reducing operational overhead.
Option C: AWS Lambda, Amazon Kinesis Data Firehose, and Amazon Athena
- AWS Lambda: Lambda can be used for small, serverless...
Author: Isabella · Last updated Jul 30, 2026
A company uses an AWS Lambda function to transfer files from a legacy SFTP environment to Amazon S3 buckets. The Lambda function is VPC enabled to ensure that all communications between the Lambda function and other AVS services that are in the same VPC environment will occur over a secure network.The Lambda function is able to connect to the SFTP environment successfully. However, when the Lambda function attempts to upload files to the S3 ...
Let's evaluate the options based on the requirement to resolve the timeout issue for the Lambda function when uploading files to Amazon S3. The Lambda function is VPC-enabled and is encountering timeout errors while trying to upload to S3, indicating a networking issue between the Lambda function and S3.
Option A: Create a NAT gateway in the public subnet of the VPC
- NAT Gateway: A NAT gateway allows instances in a private subnet to connect to the internet. While a NAT gateway can help route traffic to the internet, it introduces a cost, as it is a managed service that charges for usage (both for data processing and traffic).
- Relevance: The Lambda function is trying to access Amazon S3, which is a public AWS service. Using a NAT gateway for this scenario adds unnecessary complexity and cost because it is designed for outbound internet access for private subnet resources, not specifically for AWS services like S3.
This option is rejected because it’s not the most cost-effective solution and introduces unnecessary complexity and charges for the NAT gateway.
Option B: Create a VPC gateway endpoint for Amazon S3
- VPC Gateway Endpoint: A VPC gateway endpoint is a secure and cost-effective way to connect a VPC directly to Amazon S3 without the need for a public internet route. It allows resources in the VPC to access S3 over the AWS backbone network rather than the public internet.
- Relevance: The Lambda function needs to upload files to S3, and a VPC gateway endpoint ensures that traffic to and from S3 remains within the AWS network, which is more secure and avoids timeout issues that could arise from routing through the internet. This is the most cost-effective option because it doesn't involve any data transfer charges outside of the AWS infrastruct...
Author: Daniel · Last updated Jul 30, 2026
A company reads data from customer databases that run on Amazon RDS. The databases contain many inconsistent fields. For example, a customer record field that iPnamed place_id in one database is named location_id in another database. The company needs to link customer records across different databas...
Let's evaluate the options to determine which solution provides the least operational overhead while meeting the requirements for linking customer records across different Amazon RDS databases, especially when fields do not match.
Option A: Create a provisioned Amazon EMR cluster to process and analyze data in the databases. Connect to the Apache Zeppelin notebook. Use the FindMatches transform to find duplicate records in the data.
- Amazon EMR: EMR is a powerful solution for processing large datasets, and it can use tools like Apache Spark and Hadoop. However, managing an EMR cluster requires a significant amount of operational overhead, including cluster provisioning, management, and scaling.
- Apache Zeppelin: While Zeppelin can be used for interactive data processing, this also requires setup and management.
- FindMatches: The FindMatches transform is a good tool for identifying duplicates, but combining it with EMR adds complexity that may not be necessary for this use case.
This option is rejected because it involves too much infrastructure management (EMR cluster) for relatively straightforward data transformation tasks, making it less ideal in terms of operational overhead.
Option B: Create an AWS Glue crawler to crawl the databases. Use the FindMatches transform to find duplicate records in the data. Evaluate and tune the transform by evaluating the performance and results.
- AWS Glue: AWS Glue is a fully managed ETL service that simplifies data preparation, transformation, and loading. A Glue crawler automatically discovers and catalogs data in the databases.
- FindMatches Transform: The FindMatches transform in AWS Glue provides a managed, serverless solution for deduplicating and matching records, making it highly suitable for this task. It abstracts much of the complexity involved in data matching and linking.
- Operational Overhead: AWS Glue is fully managed, meaning there’s no need to handle cluster provisioning, scaling, or infrastructure. It's a cost-effective and less operationally heavy solution for the task at hand.
This option is selected because it leverages AWS Glue’s fully managed service for discovering and transforming data with minimal operational overhead. The FindMatches transform is directly suited for iden...
Author: GlowingTiger · Last updated Jul 30, 2026
A finance company receives data from third-party data providers and stores the data as objects in an Amazon S3 bucket.The company ran an AWS Glue crawler on the objects to create a data catalog. The AWS Glue crawler created multiple tables. However, the company expected that the crawler would create only one table.The company needs ...
In order for the AWS Glue crawler to create only one table, the key factor is ensuring that the data schema and structure across all objects is consistent. Let’s analyze the options based on this requirement:
A) Ensure that the object format, compression type, and schema are the same for each object.
- Reasoning: The AWS Glue crawler uses metadata (schema, object format, and compression type) to determine how to catalog the data. If all objects have the same format, compression type, and schema, the crawler can recognize that all the objects belong to the same dataset and create a single table.
- Why selected: This option ensures full consistency across all data objects, which is critical for the crawler to group all objects into one table. If these aspects are consistent, the crawler will be able to handle the objects as a single logical unit.
B) Ensure that the object format and schema are the same for each object. Do not enforce consistency for the compression type of each object.
- Reasoning: Ensuring that the object format and schema are the same is important for creating a single table, but inconsistency in the compression type may still cause the AWS Glue crawler to treat objects as distinct entities, potentially resulting in multiple tables.
- Why rejected: While it is true that object format and schema consistency are crucial, varying compression types could still cause the AWS Glue crawler to treat objects differently and create multiple tables. This solution could work in some scenarios but is less reliable than option A.
C) Ensure that the schema is the same for each object. Do not enforce consistency for the file format and compression type of each object.
- Reasoning: Schema consistency is indeed important, but if the file format or compression type varies, the crawler m...
Author: Alexander · Last updated Jul 30, 2026
An application consumes messages from an Amazon Simple Queue Service (Amazon SQS) queue. The application experiences occasional downtime. As a result of the downtime, messages within the queue expire and are deleted after 1 day. The message deletions c...
To minimize data loss in this scenario, we need to ensure that messages are preserved in the queue long enough to be processed, even if the application experiences downtime. Let's evaluate each option to determine the best solutions for minimizing data loss.
Option A: Increase the message retention period
- Message Retention Period: The retention period determines how long messages are kept in the SQS queue before they are automatically deleted. By default, the retention period is 4 days, but it can be extended up to 14 days. If messages are deleted after 1 day due to downtime, increasing the retention period would allow more time for the application to process messages before they are removed.
This option is selected because increasing the retention period ensures that messages are not deleted prematurely, providing more time for the application to consume them and reducing the risk of data loss during downtime.
Option B: Increase the visibility timeout
- Visibility Timeout: The visibility timeout controls how long a message remains invisible to other consumers after it has been received by a consumer. Increasing the visibility timeout prevents other consumers from attempting to process the message while it is being worked on. However, it does not directly address the issue of messages being deleted due to expiration. If the application is down for an extended period, increasing the visibility timeout does not prevent the message from expiring in the queue.
This option is rejected because it does not impact the message retention period or prevent messages from expiring if the application is not processing them in time.
Option C: Attach a dead-letter queue (DLQ) to the SQS queue
- Dead-Letter Queue (DLQ): A DLQ is used to capture messages that cannot be successfully processed after multiple attempts. If a message cannot be processed (e.g., due to application downtime), it is moved to the DLQ. While this helps handle undeliverable messages, it does not prevent messages from being deleted due to expiration if they a...
Author: Elizabeth · Last updated Jul 30, 2026
A company is creating near real-time dashboards to visualize time series data. The company ingests data into Amazon Managed Streaming for Apache Kafka (Amazon MSK). A customized data pipeline consumes the data. The pipeline then writes data to Amazon Keyspaces (for Apache Cassandra), Amazon OpenSearch Service...
To determine the best solution, we must focus on achieving the lowest latency for visualizing the data in real time. Let's analyze each option:
A) Create OpenSearch Dashboards by using the data from OpenSearch Service.
- Reasoning: This solution involves using OpenSearch Service, which is designed for search and real-time analytics on large datasets. Since the data is already written to OpenSearch Service, OpenSearch Dashboards can immediately visualize the data with minimal latency.
- Key factors: OpenSearch Service is well-suited for time-series data and near real-time analytics. The latency between ingestion and visualization will be minimal because it leverages OpenSearch, which is optimized for search and analytics workloads.
- Why this is good: Real-time visualization is directly supported with OpenSearch Dashboards. The data is already in OpenSearch, so there is no need for additional steps like querying or cataloging.
- Why other options are rejected:
- Option B and C involve querying Amazon S3, which introduces potential latency compared to querying a service like OpenSearch that is optimized for low-latency search queries.
- Option D involves using S3 Select, which can introduce overhead due to the need for cataloging and querying from S3.
B) Use Amazon Athena with an Apache Hive metastore to query the Avro objects in Amazon S3. Use Amazon Managed Grafana to connect to Athena and to create the dashboards.
- Reasoning: Athena provides a serverless query service for querying data directly from Amazon S3. However, Athena is typically not the best option for real-time dashboards due to the overhead involved in querying large datasets stored in S3. Additionally, while Amazon Managed Grafana is a powerful tool for visualizations, the latency in que...
Author: Siddharth · Last updated Jul 30, 2026
A company stores petabytes of data in thousands of Amazon S3 buckets in the S3 Standard storage class. The data supports analytics workloads that have unpredictable and variable data access patterns.The company does not access some data for months. However, the company must be able to retrieve all data within milli...
To determine the best solution, we need to focus on optimizing storage costs while ensuring the data can be retrieved within milliseconds. Let's evaluate each option:
A) Use S3 Storage Lens standard metrics to determine when to move objects to more cost-optimized storage classes. Create S3 Lifecycle policies for the S3 buckets to move objects to cost-optimized storage classes. Continue to refine the S3 Lifecycle policies in the future to optimize storage costs.
- Reasoning: S3 Storage Lens provides detailed insights into storage usage and access patterns, but this option requires manual adjustments of Lifecycle policies over time, which can introduce operational overhead. Refining these policies can be cumbersome, especially at the scale of petabytes of data.
- Why this is rejected: While this approach allows for optimization over time, it involves continuous management and adjustments, making it not ideal for minimizing operational overhead. The data retrieval within milliseconds requirement is not specifically addressed by this option.
B) Use S3 Storage Lens activity metrics to identify S3 buckets that the company accesses infrequently. Configure S3 Lifecycle rules to move objects from S3 Standard to the S3 Standard-Infrequent Access (S3 Standard-IA) and S3 Glacier storage classes based on the age of the data.
- Reasoning: S3 Standard-IA is intended for infrequent access but can still offer relatively fast access times. S3 Glacier provides more cost optimization but with retrieval times that can range from minutes to hours, which may not meet the "milliseconds" retrieval requirement for certain use cases.
- Why this is rejected: S3 Glacier introduces retrieval times of minutes or hours, which wou...
Author: Aria · Last updated Jul 30, 2026
A media company wants to use Amazon OpenSearch Service to analyze rea-time data about popular musical artists and songs. The company expects to ingest millions of new data events every day. The new data events will arrive through an Amazon Kinesis data stream. The company must transform the data and then ingest ...
To determine the best solution, we need to consider minimizing operational overhead, while ensuring the data is transformed and ingested into OpenSearch Service efficiently. Let's evaluate each option:
A) Use Amazon Kinesis Data Firehose and an AWS Lambda function to transform the data and deliver the transformed data to OpenSearch Service.
- Reasoning: Kinesis Data Firehose can automatically stream data to destinations like OpenSearch Service without much operational management. By integrating an AWS Lambda function, you can apply transformations to the data before it is ingested into OpenSearch. This method allows for a serverless architecture, reducing operational overhead and managing scaling automatically.
- Why this is selected: This option has low operational overhead because Kinesis Data Firehose handles the delivery, scaling, and retry logic automatically. Lambda is highly scalable fo...
Author: Zara · Last updated Jul 30, 2026
A company stores customer data tables that include customer addresses in an AWS Lake Formation data lake. To comply with new regulations, the company must ensure that users cannot access data for customers who are in Canada.The company needs a solution that will prevent user acc...
To meet the requirement of preventing access to data for customers in Canada with the least operational effort, we need to consider the best way to restrict access while ensuring compliance. Let's evaluate each option:
A) Set a row-level filter to prevent user access to a row where the country is Canada.
- Reasoning: A row-level filter allows for restricting access based on specific row values—in this case, where the country is "Canada." This is a common approach to enforce fine-grained access control on data.
- Why this is selected: Row-level filtering directly addresses the requirement to prevent access to rows based on the country. Using AWS Lake Formation, the company can define and enforce these filters with minimal operational effort. It allows granular control over which rows a user can access based on the country, and it integrates seamlessly with Lake Formation's existing data access controls.
- Why other options are rejected:
- Opti...
Author: VenomousSerpent42 · Last updated Jul 30, 2026
A company has implemented a lake house architecture in Amazon Redshift. The company needs to give users the ability to authenticate into Redshift query editor by using a third-party identity provider (IdP).A data engineer must set up the ...
To enable users to authenticate into Amazon Redshift Query Editor using a third-party identity provider (IdP), the first step involves configuring Amazon Redshift to recognize and trust the external IdP for user authentication. Let's evaluate each option based on this requirement.
A) Register the third-party IdP as an identity provider in the configuration settings of the Redshift cluster.
- Reasoning: This option suggests registering the third-party IdP directly in the configuration settings of the Redshift cluster, which is the correct approach. Amazon Redshift supports integration with external identity providers (IdPs) such as SAML-based IdPs for user authentication. By registering the third-party IdP, the data engineer can allow users to authenticate using the IdP credentials in Redshift Query Editor.
- Why selected: This is the correct first step because Amazon Redshift allows direct integration with IdPs through its authentication settings. Once the IdP is registered, users can authenticate with their IdP credentials.
B) Register the third-party IdP as an identity provider from within Amazon Redshift.
- Reasoning: This option is almost similar to option A, but it specifies the action of registering from within Amazon Redshift, which may be interpreted as registering within the Redshift console or using APIs. However, in practice, registering the IdP is typically done via the cluster configuration, which is what option A suggests.
- Why rejected: While this is a valid step, it lacks the clear reference to configuration settings, making it less clear than option A, which more explicitly describes th...
Author: Samuel · Last updated Jul 30, 2026
A company currently uses a provisioned Amazon EMR cluster that includes general purpose Amazon EC2 instances. The EMR cluster uses EMR managed scaling between one to five task nodes for the company=E2=80=99s long-running Apache Spark extract, transform, and load (ETL) job. The company runs the ETL job every day.When the company runs the ETL job, the EMR cluster quickly scales up to five nodes. The EMR cluster often reaches maximum CPU usage, but the memory...
The company is looking to optimize the EMR cluster configuration in order to reduce the costs associated with running the daily ETL job. We need to focus on cost-effective ways to handle the workload based on the observed usage patterns (high CPU usage, low memory usage). Let’s evaluate the options based on these factors.
A) Increase the maximum number of task nodes for EMR managed scaling to 10
- Why it's not ideal: Increasing the maximum number of task nodes would result in higher costs since the cluster could scale to 10 nodes. This would be an unnecessary cost because the CPU is already maxed out while memory usage remains low. The real problem is the cluster's CPU usage, not the lack of task nodes. More nodes will not solve the issue of CPU saturation and would just add extra cost.
- Conclusion: Not a cost-effective solution.
B) Change the task node type from general purpose EC2 instances to memory optimized EC2 instances
- Why it's not ideal: The memory usage remains low (under 30%), so switching to memory-optimized instances would be overkill. Memory-optimized instances typically cost more, and since the workload doesn't require additional memory, this change would be wasteful in terms of both resources and cost.
- Conclusion: Not a cost-effective solution.
C) Switch the task node type from general purpose EC2 instances to compute-optimized EC2 instances
- Why it’s the best choice: The primary issue is that the CPU usage is consistently h...
Author: VenomousSerpent42 · Last updated Jul 30, 2026
A company uploads .csv files to an Amazon S3 bucket. The company=E2=80=99s data platform team has set up an AWS Glue crawler to perform data discovery and to create the tables and schemas.An AWS Glue job writes processed data from the tables to an Amazon Redshift database. The AWS Glue job handles column mapping and creates the Amazon Redshift tables in the Redshift database appropriately.If the company reruns the AWS Glue job for any reason, du...
The company wants to prevent the insertion of duplicate records into Amazon Redshift when rerunning the AWS Glue job. Each option involves a different approach to solving this issue. Let’s evaluate each solution and consider the best approach based on the goal of avoiding duplicates while efficiently updating the Redshift tables.
A) Modify the AWS Glue job to copy the rows into a staging Redshift table. Add SQL commands to update the existing rows with new values from the staging Redshift table.
- Why it works: This approach involves copying data to a staging table in Redshift first and then performing an update (or an upsert) on the existing records in the target table. This avoids duplicates by ensuring only new or modified records are inserted or updated in the destination table. Using a staging table is a common practice in ETL workflows as it helps in isolating and controlling the data before it is merged into the main table.
- Why it’s a good choice: This is an effective and reliable way to prevent duplicates and ensure data integrity. It allows for full control over how records are updated in Redshift (via SQL commands such as `UPDATE` or `MERGE`), which addresses the problem of data duplication.
- Conclusion: This is a solid approach to ensure records are updated without duplication, and it provides flexibility for complex transformations or deduplication logic.
B) Modify the AWS Glue job to load the previously inserted data into a MySQL database. Perform an upsert operation in the MySQL database. Copy the results to the Amazon Redshift tables.
- Why it’s not ideal: Introducing MySQL as an intermediary step adds unnecessary complexity to the workflow. The data already resides in Redshift, and transferring it through MySQL introduces an extra layer of management, which is not needed and increases both the operational overhead and the risk of errors.
- Why it’s rejected: The solution complicates the process unnecessarily, as Redshift itself is capable of handling upserts directly. MySQL doesn’t offer any significant advantage in this case, and the added step would likely increase the overall cost and mainte...
Author: Arjun · Last updated Jul 30, 2026
A company is using Amazon Redshift to build a data warehouse solution. The company is loading hundreds of files into a fact table that is in a Redshift cluster.The company wants the data warehouse solution to achieve the greatest possible throughput. The solution must use cl...
The goal is to achieve the greatest possible throughput and use cluster resources optimally when loading data into a Redshift fact table. Let's evaluate the options based on how they impact performance and resource utilization.
A) Use multiple COPY commands to load the data into the Redshift cluster.
- Why it’s not ideal: Although using multiple `COPY` commands can parallelize the data load, the overhead of multiple commands can degrade performance. Each `COPY` command is a separate transaction, and splitting the workload into multiple commands may not utilize the Redshift resources as efficiently as one well-optimized `COPY` command.
- Why it’s rejected: While this approach may offer parallelism, it is not the most optimal approach for throughput, as the overhead from multiple commands will reduce the overall performance.
B) Use S3DistCp to load multiple files into Hadoop Distributed File System (HDFS). Use an HDFS connector to ingest the data into the Redshift cluster.
- Why it’s not ideal: This option introduces additional complexity and infrastructure by moving data through HDFS before loading it into Redshift. Redshift has native tools (like `COPY`) optimized for loading data directly from Amazon S3. Using HDFS as an intermediary step only adds extra processing overhead and doesn’t provide any significant throughput benefit.
- Why it’s rejected: Adding HDFS introduces unnecessary steps in the data loading process and complicates the solution without providing an optimal increase in throughput.
C) Use a number of INSERT statements equal to the number of Redshift cluste...
Author: Ryan · Last updated Jul 30, 2026
A company ingests data from multiple data sources and stores the data in an Amazon S3 bucket. An AWS Glue extract, transform, and load (ETL) job transforms the data and writes the transformed data to an Amazon S3 based data lake. The company uses Amazon Athena to query the data that is in the data lake.The company n...
The company needs to identify matching records in a data lake where records do not have a common unique identifier. Let's evaluate each option based on how well it meets the requirement of finding matching records without a unique identifier.
A) Use Amazon Macie pattern matching as part of the ETL job.
- Why it’s not ideal: Amazon Macie is primarily designed for data privacy and security, specifically to identify and protect sensitive data, such as personally identifiable information (PII). While Macie can be useful for pattern matching in the context of data privacy, it is not suited for matching general records without a unique identifier. Macie does not have the capabilities to match records across datasets that lack a common identifier in the way that is needed here.
- Why it’s rejected: This approach does not solve the problem of matching records without a unique identifier. It’s more focused on security use cases than record matching.
B) Train and use the AWS Glue PySpark Filter class in the ETL job.
- Why it’s not ideal: The `Filter` class in PySpark is used for filtering data based on conditions, but it does not inherently provide a way to match records that don’t have a common unique identifier. Matching records without unique identifiers typically requires more sophisticated logic, such as fuzzy matching or similarity-based matching, which the `Filter` class cannot provide directly.
- Why it’s rejected: While `Filter` can help with filtering records based on conditions, it is not designed for the complex task of matching records that lack a unique identifier.
C) Partition tables and use the ETL job to partition the data on a unique identifier.
- Why it’s n...
Author: Sophia Clark · Last updated Jul 30, 2026
A data engineer is using an AWS Glue crawler to catalog data that is in an Amazon S3 bucket. The S3 bucket contains both .csv and json files. The data engineer configured the crawler to exclude the .json files from the catalog.When the data engineer runs queries in Amazon Athena, the queries also process the excluded .json files. The data engineer wants to resolve this issue. The data engineer ne...
The data engineer wants to ensure that the excluded `.json` files are not processed by Athena queries, while maintaining access to the `.csv` files. Let’s evaluate each option and see which provides the shortest query times and solves the issue efficiently.
A) Adjust the AWS Glue crawler settings to ensure that the AWS Glue crawler also excludes .json files.
- Why it's not ideal: The data engineer has already configured the crawler to exclude `.json` files, but the issue is that the `.json` files are still being processed by Athena queries. The AWS Glue crawler settings impact the cataloging process, but Athena queries may still access files directly from S3. This approach would only affect the cataloging, but it doesn't resolve the issue with Athena queries still processing the `.json` files.
- Why it's rejected: While adjusting the crawler may improve cataloging, it does not directly address the query processing issue in Athena.
B) Use the Athena console to ensure the Athena queries also exclude the .json files.
- Why it's not ideal: Athena queries work on data in S3 based on the catalog created by AWS Glue, and the exclusion of files must happen at the data source level, not just at the query level. Athena itself does not have a built-in feature to "exclude" certain file types during query execution once the data is cataloged, especially if the file types are part of the same table. This approach would not provide an optimal solution for preventing `.json` files from being processed during queries.
- Why it's rejected: Athena queries would still scan the entire table and might attempt to process the `.json` files unless they are excluded from the catalog in a more definitive manner.
C) Relocate the .json files to a different path within the S3 bucket.
- Why it works: By relocating the `.json` files to a different path in the S3 bucket, the AWS Glue crawler can be configured to exclude that path, preventing Athena queries from ...
Author: Benjamin · Last updated Jul 30, 2026
A data engineer set up an AWS Lambda function to read an object that is stored in an Amazon S3 bucket. The object is encrypted by an AWS KMS key.The data engineer configured the Lambda function=E2=80=99s execution role to access the S3 bucket. However, the Lambda fu...
Let's break down the options and their reasoning.
A) The data engineer misconfigured the permissions of the S3 bucket. The Lambda function could not access the object.
- This is unlikely because the problem is specifically related to decryption, not access to the S3 bucket itself. The Lambda function's execution role has already been configured to access the S3 bucket. Since it’s specified that the issue occurs when trying to read the encrypted object, the problem is not about basic permissions to access the object in S3.
- Rejected: Incorrect because the issue is not with accessing the object in the S3 bucket itself, but with decryption.
B) The Lambda function is using an outdated SDK version, which caused the read failure.
- This is a possible cause, but it’s unlikely. Lambda functions typically use updated versions of SDKs, and unless explicitly specified, the Lambda function would use the latest SDK version. It’s more probable that the issue stems from missing permissions to access the KMS key.
- Rejected: Outdated SDKs are rarely the cause of decryption issues, especially when AWS SDKs handle encryption and decryption seamlessly.
...
Author: Andrew · Last updated Jul 30, 2026
A data engineer has implemented data quality rules in 1,000 AWS Glue Data Catalog tables. Because of a recent change in business requirements, the data engineer must edit the data quality rules.Ho...
Let's break down the options and their reasoning.
A) Create a pipeline in AWS Glue ETL to edit the rules for each of the 1,000 Data Catalog tables. Use an AWS Lambda function to call the corresponding AWS Glue job for each Data Catalog table.
- This option involves creating a pipeline and triggering AWS Glue ETL jobs for each Data Catalog table, which is complex and requires a significant amount of operational overhead. For 1,000 tables, the number of jobs and the complexity of managing them would be high, making this solution not optimal in terms of efficiency.
- Rejected: High operational overhead due to the need to manage multiple AWS Glue jobs for each table.
B) Create an AWS Lambda function that makes an API call to AWS Glue Data Quality to make the edits.
- This option directly addresses the need to edit data quality rules for all the tables by utilizing a Lambda function to call AWS Glue Data Quality APIs. This is efficient because AWS Lambda can process API calls in bulk or in a loop without needing to manage individual jobs. It also scales well for 1,000 tables, providing a low-overhead, serverless approach to make the required changes.
- Selected: This is t...
Author: VioletCheetah55 · Last updated Jul 30, 2026
Two developers are working on separate application releases. The developers have created feature branches named Branch A and Branch B by using a GitHub repository=E2=80=99s master branch as the source.The developer for Branch A deployed code to the production system. The code for Branch B will merge into a master branch in the following week=E2=80=...
Let's break down the options and analyze the best command for the developer working on Branch B.
A) `git diff branchB master; git commit -m`
- This command would display the differences between Branch B and the master branch, but it does not handle synchronization or updating of Branch B with the latest changes from master. Additionally, it’s followed by `git commit -m`, which would suggest that you would be committing something to Branch B, but this doesn't seem necessary or relevant in this context.
- Rejected: This command is not helpful for synchronizing Branch B with master before creating a pull request.
B) `git pull master`
- `git pull master` fetches changes from the remote master branch and merges them into the current branch. However, this is not the most optimal approach because it can result in a merge commit, which could lead to a messy history. It’s better to use a rebase to make the history cleaner and linear, especially when preparing a pull request to master.
- Rejected: Merging master into Branch B is not ideal because ...
