Write your AWS DevOps tool in Haskell for Great Good
Let's just simply compare AWS CLI, Haskell Amazonka, and official Go SDK with the following very simple tasks:
List files in one S3 bucket
AWS CLI https://docs.aws.amazon.com/cli/latest/reference/s3/ls.html
aws s3 ls s3://mybucket --recursive --page-size 100
Haskell Amazonka https://hackage.haskell.org/package/amazonka-s3-1.6.1/docs/Network-AWS-S3-ListObjects.htm
send $ listObjects "mybucket" & loMaxKeys ~? 100
Of course the Haskell code is pure and lazy, so
send
just returns a data that describes the command and produce no side effect, to actually exec the command we can create a functionawsRun
to execute the command:awsRun = runResourceT . runAWS (newEnv Discover & envLogger .~ newLogger Info stdout) . within Sydney
It is similar to the step that Go SDK need to create a session first:
sess := session.Must(session.NewSession(&aws.Config{ Region: aws.String(endpoints.ApSoutheast2RegionID), }))
Go SDK https://docs.aws.amazon.com/sdk-for-go/api/service/s3/#S3.ListObjects
svc := s3.New(session.New())
input := &s3.ListObjectsInput{
Bucket: aws.String("mybucket"),
MaxKeys: aws.Int64(100),
}
result, err := svc.ListObjects(input)
if err != nil {
...
List ALL FILES in a bucket
This time we need all, not just one page of files, let's compare
AWS CLI
Can't really do it…if there are more than 1000 files 🤷♂
–page-size (integer) The number of results to return in each response to a list operation. The default value is 1000 (the maximum allowed) https://docs.aws.amazon.com/cli/latest/reference/s3/ls.html#options
Haskell Amazonka
paginate $ listObjects "mybucket" & loMaxKeys ~? 100
Simply replace send
with paginate
, amazonka will automatically pull objects into a Conduit
https://github.com/snoyberg/conduit
stream, you can simply map over the stream, .| takeC 3
to go over only first 3 pages, or even .| sinkList
to load the very large list into memory.
Go SDK https://docs.aws.amazon.com/sdk-for-go/api/service/s3/#S3.ListObjectsPagesWithContext
objects := []string{}
err := svc.ListObjectsPagesWithContext(ctx, &s3.ListObjectsInput{
Bucket: aws.String(myBucket),
}, func(p *s3.ListObjectsOutput, lastPage bool) bool {
for _, o := range p.Contents {
objects = append(objects, aws.StringValue(o.Key))
}
return true // continue paging
})
if err != nil {
panic(fmt.Sprintf("failed to list objects for bucket, %s, %v", myBucket, err))
}
fmt.Println("Objects in bucket:", objects)
Go SDK uses callback function to go over pages, it is less efficient then stream and more verbose.
Query a DynamoDB table
AWS CLI https://docs.aws.amazon.com/cli/latest/reference/dynamodb/query.html
aws dynamodb query \
--expression-attribute-values file://put-a-json-here \
--key-condition-expression "Artist = :v1" \
--projection-expression "SongTitle" \
--table-name Music
Haskell Amazonka
send $ query "Music"
& qExpressionAttributeValues .~ HashMap.singleton ":v1" (attributeValue & avS ?~ "No One You Know")
& qKeyConditionExpression ?~ "Artist = :v1"
& qProjectionExpression ?~ "SongTitle"
Go SDK
svc := dynamodb.New(session.New())
input := &dynamodb.QueryInput{
ExpressionAttributeValues: map[string]*dynamodb.AttributeValue{
":v1": {
S: aws.String("No One You Know"),
},
},
KeyConditionExpression: aws.String("Artist = :v1"),
ProjectionExpression: aws.String("SongTitle"),
TableName: aws.String("Music"),
}
result, err := svc.Query(input)
if err != nil {
...
You should get the idea by now.
I guess anyone even can't read Haskell at all can identify the Haskell version is basically the same as AWS CLI, with some simple syntax mapping you can instantly translate any AWS CLI command into Haskell code.
--
to&
- kebab-case to CamelCase
- connect option name and value with
.~
instead of space, or?~
when the option is optional