I’m trying to list files in a particular folder in my S3 bucket using CloudStreams S3 connector but its not working if there is “/” in prefix parameter (key name).
Here if I use only “POC” in prefix, or no prefix at all, it works fine and listing files. But as soon as I get into subfolder and add a “/” in prefix, I get this error.
I tried with a “\” (backslash) and “%2F” in place of slash, that way I get a 200(OK) response, but no file list in body. So I’m sure it has to do something with “/”, but no idea what.
Any Suggestion?
Error: The request signature we calculated does not match the signature you provided. Check your key and signing method.
Do the objects have to be created with a leading slash? In a traditional file system, the leading slash represents the root directory but in S3 it has no special meaning. If you could avoid creating the object with a leading slash, that would likely be your best bet. As you can see from the examples in this page, object keys don’t typically start with a leading slash: Organizing objects using prefixes - Amazon Simple Storage Service. In fact, when I tried to create an object using CloudStreams with a leading slash in the key, the leading slash was removed (either by the connector or S3) when I looked up the object in S3.
You are right in S3 slash isn’t necessary. But here I am making same use of slash as in other file systems. There will be parent folder, further could be client wise folders, business segments, etc.
You can use operation getBucket to get all the objects in the bucket. For those folders in the bucket, the content will have the variable key and size. Folder object usually ending with “/” and of size=0. Just iterate through the content and get what you want.
