Mturk Upload New Worker Ids to Account
Tutorial: How to create HITs that ask Workers to upload files using Amazon Cognito and Amazon S3
Amazon Mechanical Turk (MTurk) serves 2 audiences — Requesters and Workers. Requesters create tasks (called Man Intelligence Tasks, or HITs for short) and publish them for Workers to consummate. These tasks can involve a latitude of questions and activities for Workers such as categorizing, labeling, annotating, rating, judging, and more. Sometimes Requesters have tasks where they demand Workers to upload as some or all of the Hitting submission. In today'south tutorial, nosotros're going to step through one mode of accomplishing this using Amazon Web Services (AWS).
In this tutorial we will step through ane style of doing this using Amazon Spider web Services (AWS). We volition create a task that allows Workers to upload files direct to your Amazon S3 business relationship. We will use Amazon Cognito and AWS Identity and Access Management (IAM) to implement this deeply.
Setting upwards Amazon Cognito
Begin past logging into the AWS Console. In one case yous've signed in, type Cognito in the search box and click to load it:
You'll exist taken to a screen similar to the i shown below. Click Manage Identity Pools here:
If you haven't created an Identity Pool before, yous'll be asked to fill in a pool proper name (e.g., "mturk_file_upload_pool") and yous'll desire to make sure the checkbox to Enable access to unauthenticated identities is checked. It should expect similar this:
Then click Create Pool. Next, Cognito will automatically create IAM roles for Unauthenticated identities and identities that come via Authentication providers (which we won't be using in this tutorial). The screen should look like this:
You'll want to store the function proper name for unauthenticated identities since we'll be using it later on. In this tutorial, that office name is: Cognito_mturk_file_upload_poolUnauth_role. Click Allow and go to the next step.
On the next screen, y'all'll be shown some example code. We're going to provide the code for this tutorial here, so you can ignore that. But you volition want to save your Identity puddle ID (in the screenshot below, the Identity puddle ID is u.s.-east-1:c286a04b-7222–4db2-a929–61478cfd55d4). The screen should look like this:
Creating your Saucepan in Amazon S3
Now, allow's create a place where we'll receive the file uploads. To exercise this, become back to the Console and select the Amazon Simple Storage Service, or S3, from the aforementioned search field nosotros entered Cognito in earlier. When information technology appears, select it to open the S3 Panel.
From here, click + Create bucket to create a new bucket.
In this tutorial, we'll create a new bucket chosen mturk-worker-uploads. Considering all bucket names are globally unique, you'll need to select a different proper noun. Whatever proper name you cull, please make sure to go along it handy. We'll be using it once again shortly.
Then click Next and select "Versioning" to enable this setting. You should see a screen like this where y'all'll select Enable versioning. By enabling versioning, you brand certain that if ii Workers upload a file with the same name, both copies will be preserved.
Then click Relieve. Go on to click Next until you become to the final screen, where y'all'll click Create saucepan. It should look like this:
Y'all demand to make one more than change to your S3 bucket. Yous'll setup Cantankerous-Origin Resource Sharing (CORS) permissions. To do that, observe the newly created S3 saucepan, click it, and select the "Permissions" tab. You should run across a screen like this:
Now select "CORS configuration" and paste the following code in to replace what is already nowadays in the CORS configuration editor:
<!-- Sample policy -->
<CORSConfiguration>
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>Post</AllowedMethod>
<AllowedHeader>*</AllowedHeader>
<ExposeHeader>ETag</ExposeHeader>
</CORSRule>
</CORSConfiguration> That's information technology, now you lot've created the bucket where we'll store the files that Workers upload.
Giving your Cognito role permission to your S3 bucket
Y'all've created your Identity Pool in Amazon Cognito, which is going to allow us make calls to AWS without having to betrayal your private credentials publicly. Y'all've also created your bucket in Amazon S3, which volition allow u.s. to store the uploaded files. Now, we're going to make certain that your Cognito roles accept access to store the files in S3. In fact, we're going to limit this role to only uploading files, and to only upload files to the specific bucket we created for this purpose. To do this, we'll utilize AWS Identity and Access Management (IAM) roles.
It turns out, y'all've already used IAM. In the second-to-last step of setting upwardly Cognito, we created ii Identity and Admission Management (IAM) roles: one for authenticated users (which we aren't roofing in this tutorial), and another for unauthenticated users (which is what the MTurk Workers volition be in this case).
Side by side, nosotros're going to make sure our IAM role for unauthenticated users has the correct permissions. To do that, let's go back to the console and search for IAM.
From in that location, click Roles on the left side of the screen, then search for the IAM function that Cognito created for us earlier. For this tutorial, we merely need to alter the "unauth" function created for unauthenticated users. In our case, this name was "cognito_mturk_file_upload_poolUnauth_role"
One time you observe the function, click it to open up information technology. It should await like this:
Side by side, click + Add inline policy in the bottom correct corner of the screen. A policy is simply a description of what permissions this role will have. We're going to use the Visual Editor to help us create our policy.
To begin, click Cull a Service and select "S3" from the listing. So, under Deportment we are going to select "PutObject" and "PutObjectACL" which volition give permission to Workers to upload (or PUT in S3 terminology) files. Your screen should look like to this:
Nether Resources, we will define a Specific resources, and click Add together ARN to exercise that. Enter the following for your ARN, merely changing mturk-worker-uploads to the name you gave your Amazon S3 bucket earlier:
arn:aws:s3:::mturk-worker-uploads/* The screen should look similar to this:
Then click Add. When yous're all washed, it should look like this:
So click Review Policy to go along. You'll be taken to a screen to review the policy that you just created. It should expect like this, just with your bucket name instead of mturk-worker-uploads:
From hither, click Create Policy to proceed. Your role should at present have an boosted policy attached that looks like this:
At this indicate, you've created a Cognito identity pool to make AWS calls, an S3 saucepan to store the files uploaded, and an IAM dominion to only allow Cognito to upload, and to only upload them into the saucepan you specified.
Next, we're going to pull everything together with a HIT that handles file upload.
Creating your HIT
To create our Hitting, permit's build on a previous tutorial. In a tutorial titled "How to create HITs using our Carmine SDK and Hit editor," we demonstrated using the MTurk Requester Website together with the AWS SDKs (in this tutorial, it was with the Cherry SDK) to go the best of both worlds.
That tutorial used a situation where we'd provide Workers with a link to a eating place, and enquire that the Worker collect the restaurant'southward hours of functioning. At present that we have the ability to ask Workers to upload a file, we'll extend this tutorial to ask Workers to upload the eating place's card as well. Please annotation that in the lawmaking below, Workers volition non be required to upload a file, but volition have the pick of uploading a file. If you demand to require that Workers have uploaded a file, we suggest checking out Javascript libraries that support this such as JQuery Validation (yous can check out a demo hither).
To practice this, nosotros demand to insert some additional code. Hither'southward the new code that nosotros'll apply for your HIT layout. This replaces the "layout.html" file from the previous tutorial.
Y'all'll notice a few things are different compared with the previous tutorial's HTML layout (called layout.html). Hither'south a summary of the key differences:
<fieldset>
<label>Please upload an paradigm of the restaurant'southward menu</label>
<div sytle="padding: 10px;">
<input id="file" blazon="file" />
<push id="upload_file_button" type="button">Upload file</push>
<input id="file_uploaded" type="hidden" value="simulated" />
<span id="status"> </span>
</div>
</fieldset> In this lawmaking snippet, that we added to the chief HTML, nosotros're inserting a new fieldset that asks Workers to upload the restaurant's menu, adds the HTML file upload control, a button to trigger the upload, and a <span> to show status updates about the upload.
At the top of the file, we included two new Javascript libraries to help us out. At that place are the AWS SDK and the JQuery library. They are shown hither:
<script src="https://sdk.amazonaws.com/js/aws-sdk-2.142.0.min.js"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script> At the bottom of the file, you lot'll notice we added a <script> tag with this cake of lawmaking:
<script>
allow config = {
region: 'us-e-1',
puddle: 'united states-east-1:c286a04b-7222-4db2-a929-61478cfd55d4',
bucket: 'mturk-worker-uploads'
} AWS.config.region = config.region;
// Configure the credentials provider to use your identity pool
AWS.config.credentials = new AWS.CognitoIdentityCredentials({
IdentityPoolId: config.pool,
});let s3 = new AWS.S3({
apiVersion: '2006-03-01',
params: { Saucepan: config.saucepan },
});do_upload = role(event) {
let file = $("#file").prop('files')[0];
// Reset the condition indicator and hidden file uploaded flag
$("#status").text("Uploading...");
$(upshot.currentTarget).prop("disabled", true);
allow workerId = turkGetParam('workerId');
let assignmentId = turkGetParam('assignmentId');
let hitId = turkGetParam('hitId');
allow fileKey = hitId + '-' + assignmentId + '-' + workerId + '-' + file.proper noun;s3.upload({
Cardinal: fileKey,
Body: file,
ContentType: file.blazon,
ACL: 'bucket-owner-full-command'
}, function(err, data) {
if (err) {
warning("Failed to upload. Please contact the Requester.");
$("#condition").text("");
}
else{
$("#status").text("Success.");
}
$(event.currentTarget).prop("disabled", false);
});
};$("#upload_file_button").on("click", do_upload);
</script>
Let's step through this code to explicate it. At the peak, in this section, you'll desire to replace these values with your own from earlier in the tutorial. The region is your AWS region, the pool is your Cognito Identity pool, and bucket is the S3 bucket you setup earlier:
allow config = {
region: 'united states-due east-1',
puddle: 'u.s.a.-east-1:c286a04b-7222-4db2-a929-61478cfd55d4',
saucepan: 'mturk-worker-uploads'
} This block of code uses the identity puddle to assemble AWS credentials from Cognito:
// Configure the credentials provider to use your identity pool
AWS.config.credentials = new AWS.CognitoIdentityCredentials({
IdentityPoolId: config.pool,
}); In the main trunk of the <script> tag, we created a function called do_upload. This handles the heavy lifting to do the file upload. First, information technology creates a pre-signed S3 post past calling s3.createPresignedPost, so it performs the upload to S3 by calling s3.upload. It changes the file name to make it easier for you, the Requester, to link the file to the Worker that uploaded it. All files are uploaded to S3 in the format [hitId]-[assignmentId]-[workerId]-[filename.extension]. You can learn more about the AWS SDK for Javascript by checking out the documentation.
If yous follow the previous tutorial, but use the new file called layout_with_file_upload.html instead of layout.html from the previous tutorial, you should be able to create HITs that look like this:
Finding your files
Once Workers upload files, they'll immediately be available in your S3 bucket. In our example, we'll get to our bucket named 'mturk-worker-uploads' and with a filename of the form [hitId-assignmentId-workerId-uploadedfilename].
Wrapping upwards
And that'due south it. Congratulations! You've successfully used Amazon Cognito, Amazon Simple Storage Service (S3), AWS Identity & Access Direction, and Amazon Mechanical Turk to allow Workers to anonymously upload files directly to your S3 bucket from an MTurk Hit. Squeamish piece of work!
We hope you institute this useful. If y'all have whatsoever questions, please mail service a question to our MTurk forums. To become a Requester, sign up here. Want to contribute as a Worker customer? Go started here.
schullertince1996.blogspot.com
Source: https://blog.mturk.com/tutorial-how-to-create-hits-that-ask-workers-to-upload-files-using-amazon-cognito-and-amazon-s3-38acb1108633
0 Response to "Mturk Upload New Worker Ids to Account"
Post a Comment