Setting up Log Analysis and KVS stream with Deepracer for cloud on EC2

View of the model training.
DR_KINESIS_STREAM_ENABLE=True
model_logs_root = 'rl-deepracer-1/'
my_bucket ='deepracer-finlay-macrae'
import boto3
import os
def download_dir(client, resource, dist, local='/tmp', bucket='your_bucket'):
paginator = client.get_paginator('list_objects')
for result in paginator.paginate(Bucket=bucket, Delimiter='/', Prefix=dist):
if result.get('CommonPrefixes') is not None:
for subdir in result.get('CommonPrefixes'):
download_dir(client, resource, subdir.get('Prefix'), local, bucket)
for file in result.get('Contents', []):
dest_pathname = os.path.join(local, file.get('Key'))
if not os.path.exists(os.path.dirname(dest_pathname)):
os.makedirs(os.path.dirname(dest_pathname))
if not file.get('Key').endswith('/'):
resource.meta.client.download_file(bucket, file.get('Key'), dest_pathname)

client = boto3.client('s3')
resource = boto3.resource('s3')
download_dir(client, resource, model_logs_root, '/workspace', bucket=my_bucket)
log = DeepRacerLog(model_logs_root)# load logs into a dataframe
log.load()
try:
pprint(log.agent_and_network())
print("-------------")
pprint(log.hyperparameters())
print("-------------")
pprint(log.action_space())
except Exception:
print("Robomaker logs not available")
df = log.dataframe()

--

--

--

Virgin Money UK IT manager and AWS Deepracer Pro racer.

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

ML.NET and the magic of Model Builder using the ASL alphabet

Building a LSTM by hand on PyTorch

PID Control for self-driving

Path Planning for Autonomous Vehicles | Intellias Blog

[Part 2] Automation of Home Appliances using Raspberry Pi — Implementation of Hand Gesture…

Multi-armed Bandits: an alternative to A/B testing

Time Series and How to Detect Anomalies in Them — Part II

What is the NaïveBayes Classifier Algorithm and how does it work?

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Finlay Macrae

Finlay Macrae

Virgin Money UK IT manager and AWS Deepracer Pro racer.

More from Medium

Microsoft Sentinel Machine Learning (ML) based Threat Detection Rules

A “Practical Data Science” Approach to Detecting Meteors with CAMS

How to calculate polyarea in metres using latitude and longitude coordinates

k-means clustering and its real use case in the security domain: