TOYOTA and AWS Case Study on Collision Assistant Application

Building a scalable, affordable, secure, and high performing product using AWS

Scalability and affordability

Security

Performance

Initial architecture

  1. The Kinesis stream receives vehicle data from an upstream ingestion service.
  2. A Lambda function writes lookup data to Amazon DynamoDB for every Kinesis record.
  3. This Lambda function decreases obvious non-crash data. It sends the current record (X) to Amazon SQS. If X exceeds a certain threshold, it will remain a crash candidate.
  4. Amazon SQS sets a delivery delay so that there will be more Kinesis/DynamoDB records available when X is processed later in the pipeline.
  5. A second Lambda function reads the data from the SQS message. It queries DynamoDB to find the Kinesis lookup data for the message before (X-1) and after (X+1) the crash candidate.
  6. Kinesis GetRecords retrieves X-1 and X+1 because X+1 will exist after the SQS delivery delay times out.
  7. The X-1, X, and X+1 messages are sent to the data science (DS) engine.
  8. When a crash is accurately predicted, these results are stored in a DynamoDB table.
  9. The push notification is sent to the vehicle owner.
initial Architecture of Application

Adjusting our initial architecture for better performance

  • Our Kinesis GetRecords API exceeded the allowed five requests per shard per second. With each crash candidate retrieving an X-1 and X+1 message, we could only evaluate two per shard per second, which isn’t cost-effective.
  • Additionally, the downstream SQS-reading Lambda function was limited to 10 records per second per invocation. This meant any slowdown that occurs downstream, such as during DS engine processing, could cause the queue to back up significantly.
Collision Assistant Application

Conclusion

Thank you !! like share and keep Reading !!

--

--

--

DevOps, Cloud, MlOps, Linux

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

linuxmuster.net v7 — First Glance at the New Web Interface

Writing a Passwordless Service With Go and Docker

Static libraries

How Long Does It Take To Learn To Code?

woman coding with beginner coding books next to her on the couch

Kava-4 Asynchronous Network Upgrade

Protobuf, gRPC and Code Generation

A swift game of Chess, part I

What is Go Language? How to install it?

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Deepak Parihari

Deepak Parihari

DevOps, Cloud, MlOps, Linux

More from Medium

Understanding AWS Control Tower

Cloud Platforms like AWS and more….!!!!!

AWS IAM —  Access Key Rotation and Notification using Boto3

Top 10 AWS Services — WHY CLOUD & WHAT IN CLOUD!