In order to get your logs from a s3 bucket, we require that you will give us access to:
- "s3:ListBucket" - So we will know which files are in your bucket and can thereby keep track of which one we have already ingested
- "s3:GetObject" - So we can download your files and ingest them to your account
Below is the IAM policy required in order for logz.io to fetch logs from your bucket:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "s3:ListBucket" ], "Resource": [ "arn:aws:s3:::<BUCKET_NAME>" ] }, { "Effect": "Allow", "Action": [ "s3:GetObject" ], "Resource": [ "arn:aws:s3:::<BUCKET_NAME>/*" ] } ] }
Test your configuration
- Make sure that you have s3cmd installed
- Note that s3cmd will usually prefer your locally-configured s3 credentials over the ones that you provide as parameters. So, either backup your current s3 access settings, or use a new instance or Docker container.
- Run s3cmd -- configure and enter your Logz.io IAM user access and secret keys.
- Try to list the bucket:
s3cmd ls s3://<BUCKET_NAME>/<BUCKET_PREFIX>/
(<BUCKET_PREFIX> is optional)
5. Try to get a file from the bucket (one of those you received from the "ls" command)
s3cmd get s3://<BUCKET_NAME>/<BUCKET_PREFIX>/<OBJECT_NAME>
If both commands were successful, we should be able to pull your logs without a problem.
Important note
- Note that the ListBucket permission is set to the entire bucket and the GetObject permission ends with a /* suffix, so we can get files in subdirectories
Comments
0 comments
Please sign in to leave a comment.