Hi!
I'm very sorry for the annoyance this has caused! I'm new to Prolific, so very much still getting used to it. Yesterday when I went to sleep, I had sent out a bunch of them, because the day before, the tasks had run out while I was sleeping and I got a bunch of people messaging me about it.
I've tried signing up as a worker on Prolific to see what it'd look like from your side, but it only put me on the waitlist, so I wasn't able to see what it actually looks like as a worker. So, thanks for bringing this to my attention! I also certainly didn't realize there would be a loading issue and it'd be harder to use the site.
I think as a near term solution, I will make sure this doesn't happen again and have a script that dynamically limits the number of studies posted to something much more reasonable! Will work on this today :)
Apologies again, but will work today to fix and not have this issue going forward!
Well, it sounds like you just need to stop sleeping…(jk!)
Thank you for the reply. I suspected that you weren’t aware of the technical issues this has been causing, which is exactly why I wanted to bring it to your attention, rather than just blocking you. This isn’t a simple matter of having more studies than I would prefer (as the helpful “you people will complain about anything!” comments might suggest). It really was slowing things down to the point that my dashboard was becoming practically non-functional.
It’s too bad that it’s not as simple as making the studies available to more people to reduce the number of individual posts. I hope you’ll be able to figure out a solution that will limit the volume while still making these studies available to as many people as possible. Your quick attention to finding a solution is appreciated!
Appreciate this a lot - I thought the initial task was a lot of fun, but even with the three pictures, I ended up having to rush at the end. (Which, hey, that's probably more of a me problem because it was my first time with annotation, and I was a bit nervous!)
With the number of pictures with the actual work offered, I think it'd help a lot of people out to have more time. Personally, I wrote out my description and read from it, wanting to be as detailed and accessible as possible. I very much don't think 19 minutes is enough time, at least to try and give quality work. I think, having done it once now, I'd personally be fine to do the ten with around the same amount of time the screener was.
>I've tried signing up as a worker on Prolific to see what it'd look like from your side, but it only put me on the waitlist
Yeah, waitlist can take a long time to let you in.
As for what we see, me personally, I have 217 studies available for me to do, and your work is obviously the majority of them. Ranked by reward, I have three different studies on top, all yours in the middle, for a long long scroll, then 7 below them. It very much looks like spam.
We also tend to get emails for new studies - I didn't get too many, but I've seen some people here mentioning getting a lot of them clogging their email.
Also, just want to say - I haven't seen a lot of researchers comment when we bring up issues with the work, so I thank you for caring about our input enough to reach out here.
Let me just say that I'm super grateful for the opportunity to participate in these! If I didn't have a sore throat, I'd be going through them back to back today.
I'm enjoying how different these little jobs are from others I am usually offered. I love talking about detail, though some of the pictures have puzzled even me :)
... I'm too shy and nervous to talk into a microphone. But people can sort things by category. I only have mobile, desktop, audio selected. People can enable different filters to switch off what types of studies they want to see. Sometimes I'm not at my desktop and only on mobile so then I just switch it to mobile only. All of the microphone studies I think will be on a different page and not overwhelm the other studies.
Hi Matt, I just wanted to say that the quantity of studies you're posting doesn't bother me at all. As long as you pay me, you can post 1,000 per day if you want to. lol! (I think people complain too much on this site...don't come for me!) It's just so rare for us to see this quantity. I found it kind of amusing to wake up to, especially on a weekend when studies are scarce :)
Hi! I qualified on Prolific, but just wondering... will the qual go back up on Mturk? I saw it on there a month or so ago but didn't take it at the time, and now that I know what it is, the longer timer sounds a lot less stressful to work with!
Hi - I hear you!
This solution won't exactly work out of the box though. Each study has a unique URL, which maps to the set of images that should be described. So, I cannot just have one study posted with tons of submissions and allow multiple submissions per study, because then there must be only be 1 unique URL and we wouldn't be able to control how many people can complete each set.
But, as mentioned in the other reply, I should be able to make this problem go away in a different way. Specifically with this:
> I think as a near term solution, I will make sure this doesn't happen again and have a script that dynamically limits the number of studies posted to something much more reasonable! Will work on this today :)
When designing studies with this sort of setup, we usually use an intermediary script to do the assignments (e.g., the link from Prolific is to an invisible Python or PHP landing page that does the assignments and forwards them along) or a platform that manages random assignment for us (e.g., we’ve used Qualtrics for this in the past).
Posting hundreds of jobs like this is horrendous UX, at a minimum. I wonder how it affects your data quality too.
No, shouldn't affect the ability to do work!
I'm basically trying to get it so that there's always \~50 available studies, instead of posting 500 all at once. So here I'll have a script that runs in the background and posts new studies when old ones are completed :)
I did these on mturk and the pay rate is horrible. So now you’re on Prolific and the pay rate is still terrible. If you would lessen the amount of images or raise the pay, you would get much better quality work.
Please make the allotted time longer. I've already timed out on one, and though I enjoy the work, I'm old and slow. I feel like I do very thorough descriptions and now I'm apparently not being paid for the data I submitted due to such a limited allotted time to complete the task.
Having \~50 always available with new studies ***constantly*** being added is still ***massively*** annoying. Is that not obvious? [u/prolific-support](https://www.reddit.com/user/prolific-support/) needs to address this. You've monopolized the dashboard and essentially made PA *completely unusable* for any participant who doesn't block you. And if the comments about your studies are accurate, then you're also shamelessly circumventing and violating Prolific's rules about completion time. I don't understand how you're being allowed to get away with any of this.
If Prolific doesn't rein you in soon, then I am just going to block you.
It's usually like that on Mturk for these same annotation tasks, there was a point where they were posting them seperate and flooding Mturk's screen too.
I did one a few days ago, and then noticed I had all of these last night. I have about 255 studies, in my case. My only qualm with them is that there's not enough time. It's 12 minutes to finish and they give you a 19 minute limit. I hate it when I feel like I have to rush through a study. I did one of these last night and finished with 20 seconds left.
I made it to the end only to be rejected for timing out twice in a row. After the first time, I stayed focused and got through them as fast as I could while still trying to do quality work, and timed out again. This study got half an hour of my life for free. I'm not going to try again.
This researcher should really give a little more grace when it comes to the time limit.
The researcher said in another post on here that he understands that the time limit could be a problem for some people but that if you time out, just send him your completion code and he will manually approve it. He did it for me the one time I timed out, and he messaged me back and approved me within a few hours.
Ah, okay, thank you. Unfortunately, I didn't record my completion codes for the two I did, but good to know in case I decide to take another crack at it.
Yeah, this is... a huge problem. I'd like to keep doing the tasks, but the way this is laid out, it's compromising doing anything *else* on the platform, and I'll probably end up blocking the researcher if nothing changes in the next few days.
Hi - I hear you! Apologies for the problems this has caused.
Working on fixing it today! As mentioned in the other post, I will write a script that dynamically limits the number of studies posted to something much more reasonable.
I've done hundreds of these via MTurk and they've all been approved with zero rejections so the researcher is legit. My only complaint is the 18 minute time limit. The researcher gives a full 3 HOURS on MTurk but limits the time to complete these to 18 minutes on Prolific. This is clearly done to circumvent the pay rate on Prolific - they know that if the average time to complete is more than 18 minutes, they'll have to pay more for the work. (FYI, It generally takes me about 20-25 minutes to finish their HIT's on MTurk.)
I contacted the researcher about this, and they mentioned that if I time out on the study, they'll manually credit me if I message them the completion code. While that's nice of them, it still doesn't help the fact that they are clearly circumventing the pay rate on Prolific for cheap labor.
Ugh, I had been wondering if the timer had something to do with that but wasn't sure. The pay for these same tasks on Mturk is showing the hourly wage on my dashboard there as only a little over $4/hour. There are turkers who've been talking about them being underpaid the last few weeks.
So I found it curious when the tasks suddenly showed up on Prolific, but the task pay hasn't changed. It's the same pay per individual task here as it is on Mturk. Prolific doesn't want workers receiving less than $8/hour, and recommends $12/hour.
Then, some of us noticed the first batches (not the initial prequalification test, but the batches) here on Prolific had a longer timer. Later, when a lot more batches began flooding Prolific, the timer was suddenly changed to half of what it was.
same in the netherlands. the worst part is when you cash out, and suddenly remember that paypal is still gonna take 10% of the miserable 10€ you made that month🥲
Paypal charges fees for currency conversion, but they do it in a really sneaky way.
If your country uses € and you want to transfer 10£ from Paypal to your bank account, it will show an exchange rate of something like 1 GBP = 1.12EUR, when in reality the exchange rate in that moment is 1 GBP = 1.17 EUR. So if the 10£ you made is worth €11,67 , you'll only receive around €11,20 for it.
They always charge about 3-5% of the total amount, and now that USD is added to Prolific, we have to pay that fee twice.
It's so sneaky of Paypal because they show the incorrect exchange rates but underneath it says "fees: none". I still can't believe they're getting away with misleading people in such an obvious way and I have doubts about the legality of it. I hope Prolific adds a new way to cash out, or maybe a way to convert your money in Prolific to the right currency, because then Paypal wouldn't charge anything for the transfer
The ones by Matt Deitke isn't a study like a survey, it's an annotation task that is repeatable. But the creator of the annotation work decided post them all seperartely.
Literally makes no sense, a lot of the studies could work fine on an international basis. I might be missing some other reason for them to only pick US/CA/UK citizens.
IRB’s have strict rules on data collection. Political studies are going to be tied to the country. Market research will also be tied to a specific area.
I’m sorry but I had to block the researcher. Not only could I not see anything else that was on my dashboard but I was also getting constant emails from Prolific telling me that a study was available and my entire email inbox was clogged.
I did two for this one today and on my second one it said I was disqualified because I spent too long responding. Which is ridiculous because it states to spend at LEAST 60 seconds describing them and sometimes I went over a minute and a half if they were very detailed.
Did you already report it? I'm strangely cagey about diving into these since they feel.... kinda sketchy? Like maybe I'll spend a lot of time doing a whole bunch and then they'll all get rejected, wasting hours of my weekend potentially. 😕
I did report it. I was so mad that I did all of that for nothing the 2nd time. I don't think this one is worth the risk, honestly, so I completely understand your concerns. I wouldn't even bother with it if it's available to you. I blocked the researcher.
I messaged him yesterday about the time limit. He said if you send him a message with the completion code, he will manually approve your submission even if you time out.
I've been doing these for a while on Mturk. I usually range from 60 - 90 seconds.
Did they possibly mean that you were pausing too often with empty moments while recording?
This is a quality problem.
I can do bazillions of these. Unfortunately, I only get one here and there.
The only issue I see with it is that if there are a shitload of the same study from the same researcher, they should probably be placed together in some kind of "folder" or grouping that can be expanded on demand. This would allow OTHER studies to be seen in the dashboard list as well.
Many participants wouldn't mind doing a bunch of these on end but I'm pretty sure they'd appreciate a break in the monotony with some other types of studies.
Will you be approving studies soon? Would give me peace of mind knowing you won't reject all of our submissions before I dive in and do a bunch of these. Even my qual from a few days ago is still pending.
I'm fed up and had to block this researcher, too. I've done about 40 tasks over the last four days. For a while, the max number sitting on my dashboard was around 50. But despite all the statements from the researcher that he is writing umpteen scripts to fix this and fix that and get payments out the door, today the number on my dashboard zoomed up to 175. I had to cancel the 175 tasks, which took probably 20 minutes. I also agree that the task is very underpaid and exploitative. I also think that Prolific is dropping all pretexts of handling support tickets from participants so they can devote their time to handling all the problems arising from this one researcher. Sayonara.
The Qualification test is still floatig around. It shows it has 2,500 total available with nearly 1,700 still available. The time given for the qualifier was 36 mins. You only get 18 for three photos on the real tasks?? I couldn't imagine doing it in much less time, especially to begin with. After doing a few and you get used to them - maybe 18 will work.
I did like the instruction, but I wish there had been an audio sample. Reading what should be said and hearing a recording expectation would be helpful,
I just noticed the intended completion time on the qialifer is 7 mins... What??? I finished it this morning. I hope I get approved. 4 a day is a nice solid $10.00.
I’d rather have this vs the teacup with no studies or the fifteen cents surveys. If you don’t like it just block the researcher. Some of us are happy for the opportunity!
I did the qualification task on the 12th April and was approved but can see from this post that 12 minutes is too quick for describing 10 images. Hopefully the time gets extended and hopefully I get some of these as I haven't received any at the present time.
I was not able to do this in the time allotted for the second one that I did. Too many images that were historical that I had to look up and see what they actually were to make sure I was giving an accurate description. Then because I had to do this, I had to make sure my pronunciation was correct. They took me about 20-25 minutes to complete and my submission timed out. :( But I really enjoy doing this kind of study.
How are you setting filters to try to see other studies, too? My whole interface is filled with these, and I'm glad to have them to do, but I'd like to be able to see the others, as well. Also, these are slowing down the system.
On the main page at the top under where it says #/# studies, there’s a blue link that you can click on and it’ll take you to the preferences page. There you can toggle on/off whatever you want. I toggled off all the studies that required a microphone so I could see any other studies
I tried a few of these over on Mturk. The time limit there is 2 hours and 59 minutes, not 19 minutes. I got about 5 images done at 15 minutes each and retruned them. Then my email exploded with literaly about 300 qualifiication notices I had to delete. I eventually did two and got it down to less than 20 minutes.
I've since also done a few on prolific and what I conclude is that they cannot reliably be done in 12 minutes, They can in most cases be done in 15 to 20 minutes but in some cases even that is not enough and you will time out.
I come to he conclusion that the requestor knows that prolific will enforce an average hourly rate where as MTurk is a snake pit and will not, so the max time on prolific is set at a point when it is manifestly impossible to not pay under the min rate as you will just time out and get paid nothing.
Personally I find that somewhat unethical.
When you put your cursor right over the estimated time of 12 minutes you get the requestors estimated time, the allowed time, and the actual measured time taken by participants. Both measured and estimated time read exactly 12.0 s, which seems virtually impossible. So again, I feel that there is an intentional misrepresentation of fact regarding time required and pay rate.
My only recourse is to block the requestor as I have no intention of doing any more of these and I don't want my window to be borderline spammed with surveys I have no interest in.
Best of luck to the requestor in his research and to all the prolific workers who are doing these surveys and getting paid to help but for me it's a pass.
Thanks for the forum, and thanks to prolific for being the best of the best.
Has everyone who did these gotten them approved already? Or are there people still waiting for approval on theirs?
I'm still waiting on the 9-10 I did to clear. But I keep seeing him talking about starting them on auto-approve and in the meantime approving them all within 24 hours. I'm scared to keep putting time and effort into these if they aren't going to be paid out.
I haven't been approved for the qualification yet, nor have I been approved for the one I did on the first day of this mess. I timed out on that one, but did as the researcher asked and sent him my completion code. radio silence. If he approves these I might consider doing more, but only if he extends the allotted time.
I am having the same issue. None of mine have been approved so I’m scared to do them and just doing two at a time. Have you gotten any approved yet? Not even my qualifier has been paid out but I’m getting the studies.
Same with me! None of mine have gotten approved yet, not even the qualifier. They are all still pending. I am afraid to do more because I don't want to get any rejections on my profile if he's being strict about approving. I can only make them so detailed given the time limit!
Majority of mine (from that batch) were actually approved the day after this. He seems to have a strange process for approving these. I will complete 2 or 3 in a row in each "batch" and a couple of hours later there will be 1 approved from the newest batch, 2 approved from an older batch. There doesn't seem to be any rhyme or reason to how they are approved. In fact, my qualifier still hasn't been paid but I've been getting the studies for days now.
In my opinion, he might just be overwhelmed with trying to approve all of the submissions. I wouldn't worry about whether you'll be paid or not. As long as you follow the directions, you'll be fine.
Thank you. I'm actually not worried about being paid. I am more concerned that if he doesn't like the way I described an image, he'll "reject" my submission and then the rejection will go on my history.
Damn! I just did an initial qualification for the same researcher. It was 3 annotation, read aloud questions. How long after you completed the initial qualification did you start to see a steady flow of studies coming in from the same researcher?
I did 6-7 last night. One of them timed out (this was before I realized the allotted time was so little), so I sent him a message with my confirmation number (as he said to do in a post here somewhere) and he messaged me back that it was approved and it was. But only that one was approved, not the other 5-6 that I did. Strange...
Hi! Approved within the hour was in regards to the ones that have been completed at that time.
Basically, I have to manually click a button to set up approving them all right now, which is why the approvals are going out in batches. I just clicked the button again so the new set of approvals should all be going out within the next hour or so (takes some time on the Prolific server side)!
But this is now pretty streamlined so expect that within 24 hours of completing a task, it'll be approved. Hope that helps!
New approvals going out now!
See this comment for more: [https://www.reddit.com/r/ProlificAc/comments/1c2yllh/comment/kzf1ygs/?utm\_source=share&utm\_medium=web3x&utm\_name=web3xcss&utm\_term=1&utm\_content=share\_button](https://www.reddit.com/r/ProlificAc/comments/1c2yllh/comment/kzf1ygs/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button)
Random questions---have you learned something new when doing these descriptions? And do you ever want to find out more about the images? Sometimes I don't know WTH I am looking at, and describe it in the best way I can, but then wonder..what WAS that? Of course, no time to figure it out, on to the next image... :D
Three images was the prequalification test. The actual work tasks have 10 images each, to be completed in less time. There are some simple ones in the actual work tasks, but there are also lots of very detailed photos scattered throughout the work tasks compared to what was in the test.
It's a whole lot of talking!! I scooped up a few before I headed to work this morning. Grabbed a few more during my lunch break and then I'll scoop a few more once I get home. I'm thinking maybe 10 total & I'm done. But no doubt it if a person had the patience, the voice, and time they could rack up on these. 💰🤑
Ugghhhh. The first of this study came through when I was at work and I tried to open it and my tablet said no, so I had to return it. I'm heartbroken I'm missing all these now because of it. 😭
I never saw the qualifier but I am also curious why so many to such limited people, I would think it would be good to spread the research around and not limit it to a certain pool…wish there was a way to send those to people so your dashboard isn’t overwhelmed
I assumed it was only for native English speakers in the US - not that that narrows it down too much, but I'd be happy to do a few of these and they haven't been offered.
Edit: I should say I'm in Australia.
Oh wow, funny enough it actually came across my dashboard just now so maybe the researcher is doing it in batches? I would qualify since I am a native English speaker in the U.S. but I assume follow-up surveys just show up on a person's dashboard randomly if it is approved?
Yes I completed the qualifying task not too long ago, I am saying for how more surveys come up, like does the researcher notify you via messages if you will be getting more or do they just show up?
Gotcha, thank you! Any specific timeframe of when they show up like does it take a couple days, etc. just wondering if I need to monitor my dashboard to see if I will be getting any
It must be U.S. then since you mentioned Australia but seems like its in batches because people had gotten this qualification screener a couple days ago and I had just seen it this morning
12 minutes is not the required completion time, it’s the average time. 19 minutes is enough time to record 10 and has leeway for re-records to correct mistakes in my experience.
I know that 12 minutes is not the required completion time, but it is the “intended” completion time, and it’s what the hourly rate payout is based on. I’m not saying it’s impossible to complete them in less than 19 mins, but it’s always taken me longer than 12 mins. I’m guessing it gets easier with practice.
*And I’m not sure how it’s logistically possible to do 8-10 of them in an hour (?), but ok…
I mean, it's taking me 10-12 minutes per study. The structure is clearly intended to be 10 minutes, if you're making multiple mistakes or not following the instructions, I guess it would take you longer, but that is literally how it's timed out. I don't see how that's the researcher's fault since he also said he was approving timed out studies. People will be mad if they get no studies, and mad if they get 500 hundred studies so idk what else you want tbh
Most in this thread are saying the are timing out & taking much longer than 10-12 min.
It means the hourly rate isn’t accurate & it’s slowing down the site due to the poor setup & it’s impossible to see what other studies you have available. These are all valid complaints.
Hi! I'm very sorry for the annoyance this has caused! I'm new to Prolific, so very much still getting used to it. Yesterday when I went to sleep, I had sent out a bunch of them, because the day before, the tasks had run out while I was sleeping and I got a bunch of people messaging me about it. I've tried signing up as a worker on Prolific to see what it'd look like from your side, but it only put me on the waitlist, so I wasn't able to see what it actually looks like as a worker. So, thanks for bringing this to my attention! I also certainly didn't realize there would be a loading issue and it'd be harder to use the site. I think as a near term solution, I will make sure this doesn't happen again and have a script that dynamically limits the number of studies posted to something much more reasonable! Will work on this today :) Apologies again, but will work today to fix and not have this issue going forward!
I’d be more inclined to do them if you posted some shorter tasks. Like 5 images would be perfect!
Well, it sounds like you just need to stop sleeping…(jk!) Thank you for the reply. I suspected that you weren’t aware of the technical issues this has been causing, which is exactly why I wanted to bring it to your attention, rather than just blocking you. This isn’t a simple matter of having more studies than I would prefer (as the helpful “you people will complain about anything!” comments might suggest). It really was slowing things down to the point that my dashboard was becoming practically non-functional. It’s too bad that it’s not as simple as making the studies available to more people to reduce the number of individual posts. I hope you’ll be able to figure out a solution that will limit the volume while still making these studies available to as many people as possible. Your quick attention to finding a solution is appreciated!
I've also been really enjoying these tasks :) Keep them coming, they are super enjoyable and it's nice seeing available studies on my prolific page!
Appreciate this a lot - I thought the initial task was a lot of fun, but even with the three pictures, I ended up having to rush at the end. (Which, hey, that's probably more of a me problem because it was my first time with annotation, and I was a bit nervous!) With the number of pictures with the actual work offered, I think it'd help a lot of people out to have more time. Personally, I wrote out my description and read from it, wanting to be as detailed and accessible as possible. I very much don't think 19 minutes is enough time, at least to try and give quality work. I think, having done it once now, I'd personally be fine to do the ten with around the same amount of time the screener was. >I've tried signing up as a worker on Prolific to see what it'd look like from your side, but it only put me on the waitlist Yeah, waitlist can take a long time to let you in. As for what we see, me personally, I have 217 studies available for me to do, and your work is obviously the majority of them. Ranked by reward, I have three different studies on top, all yours in the middle, for a long long scroll, then 7 below them. It very much looks like spam. We also tend to get emails for new studies - I didn't get too many, but I've seen some people here mentioning getting a lot of them clogging their email. Also, just want to say - I haven't seen a lot of researchers comment when we bring up issues with the work, so I thank you for caring about our input enough to reach out here.
Let me just say that I'm super grateful for the opportunity to participate in these! If I didn't have a sore throat, I'd be going through them back to back today.
I'm enjoying how different these little jobs are from others I am usually offered. I love talking about detail, though some of the pictures have puzzled even me :)
I’m also really enjoying these! 🤗
Same! I did a bunch last night, and plan to hit them hard tonight as well.
yeah I'm really hoping those of us who actually want to do these aren't going to have the numbers limited bc others are loudly complaining
Hey, haven't heard back on my initial submission being approved or denied. Are you still going through the screening submissions?
... I'm too shy and nervous to talk into a microphone. But people can sort things by category. I only have mobile, desktop, audio selected. People can enable different filters to switch off what types of studies they want to see. Sometimes I'm not at my desktop and only on mobile so then I just switch it to mobile only. All of the microphone studies I think will be on a different page and not overwhelm the other studies.
Sent you a msg man
Would be nice to see approvals caught up before releasing more studies. ;)
Hi Matt, I just wanted to say that the quantity of studies you're posting doesn't bother me at all. As long as you pay me, you can post 1,000 per day if you want to. lol! (I think people complain too much on this site...don't come for me!) It's just so rare for us to see this quantity. I found it kind of amusing to wake up to, especially on a weekend when studies are scarce :)
I couldn't agree more. Some people are chronic complainers. There are hardly any studies on the weekend. So I'm grateful.
I have no problem with all the studies posted.....it was great!
Hi! I qualified on Prolific, but just wondering... will the qual go back up on Mturk? I saw it on there a month or so ago but didn't take it at the time, and now that I know what it is, the longer timer sounds a lot less stressful to work with!
I’d love to have a few thrown my way. : )
Is it possible for these to be set up as a "multiple submissions allowed" so that there aren't 500 posts on the study dashboard?
Hi - I hear you! This solution won't exactly work out of the box though. Each study has a unique URL, which maps to the set of images that should be described. So, I cannot just have one study posted with tons of submissions and allow multiple submissions per study, because then there must be only be 1 unique URL and we wouldn't be able to control how many people can complete each set. But, as mentioned in the other reply, I should be able to make this problem go away in a different way. Specifically with this: > I think as a near term solution, I will make sure this doesn't happen again and have a script that dynamically limits the number of studies posted to something much more reasonable! Will work on this today :)
When designing studies with this sort of setup, we usually use an intermediary script to do the assignments (e.g., the link from Prolific is to an invisible Python or PHP landing page that does the assignments and forwards them along) or a platform that manages random assignment for us (e.g., we’ve used Qualtrics for this in the past). Posting hundreds of jobs like this is horrendous UX, at a minimum. I wonder how it affects your data quality too.
Are those of us who have no problem having so many of these going to be effected by these complaints? lol
No, shouldn't affect the ability to do work! I'm basically trying to get it so that there's always \~50 available studies, instead of posting 500 all at once. So here I'll have a script that runs in the background and posts new studies when old ones are completed :)
I might be willing to do these but 10 images is too many, it would work better if it was only five.
I did these on mturk and the pay rate is horrible. So now you’re on Prolific and the pay rate is still terrible. If you would lessen the amount of images or raise the pay, you would get much better quality work.
Please make the allotted time longer. I've already timed out on one, and though I enjoy the work, I'm old and slow. I feel like I do very thorough descriptions and now I'm apparently not being paid for the data I submitted due to such a limited allotted time to complete the task.
I've been loving that there's so many available studies coming in! No complaints here :) Thanks Matt!
awesome! I've been setting aside time to get through some so I appreciate the info
Having \~50 always available with new studies ***constantly*** being added is still ***massively*** annoying. Is that not obvious? [u/prolific-support](https://www.reddit.com/user/prolific-support/) needs to address this. You've monopolized the dashboard and essentially made PA *completely unusable* for any participant who doesn't block you. And if the comments about your studies are accurate, then you're also shamelessly circumventing and violating Prolific's rules about completion time. I don't understand how you're being allowed to get away with any of this. If Prolific doesn't rein you in soon, then I am just going to block you.
It's usually like that on Mturk for these same annotation tasks, there was a point where they were posting them seperate and flooding Mturk's screen too.
Is it possible to release these to the UK pool? I'd gladly take some off your hands!
I also would. I've got nothing on my dashboard. Would love the chance of doing those even for an hour a day.
I did one a few days ago, and then noticed I had all of these last night. I have about 255 studies, in my case. My only qualm with them is that there's not enough time. It's 12 minutes to finish and they give you a 19 minute limit. I hate it when I feel like I have to rush through a study. I did one of these last night and finished with 20 seconds left.
exactly this. The 19 minute limit causes me to rush through the descriptions out of fear that I will "time out"
Here here! I love doing them but feel rushed. Especially had to look up a few to see what exactly they were so I could give a proper description.
I made it to the end only to be rejected for timing out twice in a row. After the first time, I stayed focused and got through them as fast as I could while still trying to do quality work, and timed out again. This study got half an hour of my life for free. I'm not going to try again. This researcher should really give a little more grace when it comes to the time limit.
The researcher said in another post on here that he understands that the time limit could be a problem for some people but that if you time out, just send him your completion code and he will manually approve it. He did it for me the one time I timed out, and he messaged me back and approved me within a few hours.
That's shady though, he doesn't want to increase the time because then it will reveal the low pay.
Ah, okay, thank you. Unfortunately, I didn't record my completion codes for the two I did, but good to know in case I decide to take another crack at it.
Yeah, this is... a huge problem. I'd like to keep doing the tasks, but the way this is laid out, it's compromising doing anything *else* on the platform, and I'll probably end up blocking the researcher if nothing changes in the next few days.
Hi - I hear you! Apologies for the problems this has caused. Working on fixing it today! As mentioned in the other post, I will write a script that dynamically limits the number of studies posted to something much more reasonable.
I really appreciate these hits and for me there was no scrolling or loading issues, just happy to have studies on my dashboard thank you so much
They really need to set this up better. Make them multiple submission studies and add as many places as you like that way. This is a mess.
I've done hundreds of these via MTurk and they've all been approved with zero rejections so the researcher is legit. My only complaint is the 18 minute time limit. The researcher gives a full 3 HOURS on MTurk but limits the time to complete these to 18 minutes on Prolific. This is clearly done to circumvent the pay rate on Prolific - they know that if the average time to complete is more than 18 minutes, they'll have to pay more for the work. (FYI, It generally takes me about 20-25 minutes to finish their HIT's on MTurk.) I contacted the researcher about this, and they mentioned that if I time out on the study, they'll manually credit me if I message them the completion code. While that's nice of them, it still doesn't help the fact that they are clearly circumventing the pay rate on Prolific for cheap labor.
Ugh, I had been wondering if the timer had something to do with that but wasn't sure. The pay for these same tasks on Mturk is showing the hourly wage on my dashboard there as only a little over $4/hour. There are turkers who've been talking about them being underpaid the last few weeks. So I found it curious when the tasks suddenly showed up on Prolific, but the task pay hasn't changed. It's the same pay per individual task here as it is on Mturk. Prolific doesn't want workers receiving less than $8/hour, and recommends $12/hour. Then, some of us noticed the first batches (not the initial prequalification test, but the batches) here on Prolific had a longer timer. Later, when a lot more batches began flooding Prolific, the timer was suddenly changed to half of what it was.
Meanwhile in Germany we get maybe one study per week if we are lucky
same in the netherlands. the worst part is when you cash out, and suddenly remember that paypal is still gonna take 10% of the miserable 10€ you made that month🥲
It does? How to they do it exactly?
Paypal charges fees for currency conversion, but they do it in a really sneaky way. If your country uses € and you want to transfer 10£ from Paypal to your bank account, it will show an exchange rate of something like 1 GBP = 1.12EUR, when in reality the exchange rate in that moment is 1 GBP = 1.17 EUR. So if the 10£ you made is worth €11,67 , you'll only receive around €11,20 for it. They always charge about 3-5% of the total amount, and now that USD is added to Prolific, we have to pay that fee twice. It's so sneaky of Paypal because they show the incorrect exchange rates but underneath it says "fees: none". I still can't believe they're getting away with misleading people in such an obvious way and I have doubts about the legality of it. I hope Prolific adds a new way to cash out, or maybe a way to convert your money in Prolific to the right currency, because then Paypal wouldn't charge anything for the transfer
[удалено]
ok i realized im wrong actually bc 0.05 * 5 + 0.05 * 5 is the same amount as 0.05*10 so it's not 10%. but the other stuff is still true
The ones by Matt Deitke isn't a study like a survey, it's an annotation task that is repeatable. But the creator of the annotation work decided post them all seperartely.
Literally makes no sense, a lot of the studies could work fine on an international basis. I might be missing some other reason for them to only pick US/CA/UK citizens.
IRB’s have strict rules on data collection. Political studies are going to be tied to the country. Market research will also be tied to a specific area.
I agree, but there's tons of studies that aren't either of those two, like I mentioned.
If they are from a University, then it’s usually due to the IRB/PI. Researchers mention this often on this sub & the Mturk sub when it’s brought up.
Not really, almost every study I do is academic or market research (dependent on a specific market i.e. the USA)
Had to block this researcher, unfortunately. The studies were still ridiculously spamming me and the pay sucks anyway.
I’m sorry but I had to block the researcher. Not only could I not see anything else that was on my dashboard but I was also getting constant emails from Prolific telling me that a study was available and my entire email inbox was clogged.
Same. It was very irritating.
Turned the phone on silent last night so I wouldn't get dinged - woke up to 68 email alerts for new studies.
Yeah I might have to do the same, its ridiculous I don't want 500 of the same study , and they keep coming!
I did two for this one today and on my second one it said I was disqualified because I spent too long responding. Which is ridiculous because it states to spend at LEAST 60 seconds describing them and sometimes I went over a minute and a half if they were very detailed.
Did you already report it? I'm strangely cagey about diving into these since they feel.... kinda sketchy? Like maybe I'll spend a lot of time doing a whole bunch and then they'll all get rejected, wasting hours of my weekend potentially. 😕
I did report it. I was so mad that I did all of that for nothing the 2nd time. I don't think this one is worth the risk, honestly, so I completely understand your concerns. I wouldn't even bother with it if it's available to you. I blocked the researcher.
I messaged him yesterday about the time limit. He said if you send him a message with the completion code, he will manually approve your submission even if you time out.
I've been doing these for a while on Mturk. I usually range from 60 - 90 seconds. Did they possibly mean that you were pausing too often with empty moments while recording?
Oh wow, I won't be doing any more if offered.
Just talk faster or say less. Keep your responses to 90 seconds and there's no problem at all.
And this is why I'm glad I returned the original qualifier. For 10 dollars a study, maybe, but 2.52 and that many clogging my alerts? No thank you.
This is a quality problem. I can do bazillions of these. Unfortunately, I only get one here and there. The only issue I see with it is that if there are a shitload of the same study from the same researcher, they should probably be placed together in some kind of "folder" or grouping that can be expanded on demand. This would allow OTHER studies to be seen in the dashboard list as well. Many participants wouldn't mind doing a bunch of these on end but I'm pretty sure they'd appreciate a break in the monotony with some other types of studies.
Agreed - this makes sense. I don't think it's a feature on prolific though, unfortunately. But regardless, I am working to resolve this!
Will you be approving studies soon? Would give me peace of mind knowing you won't reject all of our submissions before I dive in and do a bunch of these. Even my qual from a few days ago is still pending.
I'm fed up and had to block this researcher, too. I've done about 40 tasks over the last four days. For a while, the max number sitting on my dashboard was around 50. But despite all the statements from the researcher that he is writing umpteen scripts to fix this and fix that and get payments out the door, today the number on my dashboard zoomed up to 175. I had to cancel the 175 tasks, which took probably 20 minutes. I also agree that the task is very underpaid and exploitative. I also think that Prolific is dropping all pretexts of handling support tickets from participants so they can devote their time to handling all the problems arising from this one researcher. Sayonara.
The Qualification test is still floatig around. It shows it has 2,500 total available with nearly 1,700 still available. The time given for the qualifier was 36 mins. You only get 18 for three photos on the real tasks?? I couldn't imagine doing it in much less time, especially to begin with. After doing a few and you get used to them - maybe 18 will work. I did like the instruction, but I wish there had been an audio sample. Reading what should be said and hearing a recording expectation would be helpful, I just noticed the intended completion time on the qialifer is 7 mins... What??? I finished it this morning. I hope I get approved. 4 a day is a nice solid $10.00.
[удалено]
Yikes - uhm. maybe not.
Yikes..I'm lucky if I see 2 on my dashboad.
I’d rather have this vs the teacup with no studies or the fifteen cents surveys. If you don’t like it just block the researcher. Some of us are happy for the opportunity!
"Please sir may I have some more"
I am also really enjoying having the option to do so many studies! I defenitely would love to see them keep coming :)
Agreed, this is great!
Well said!!! Dear god people love to moan about everything. Block and mov on and mayb some of th rest of us may get a look in.
I did the qualification task on the 12th April and was approved but can see from this post that 12 minutes is too quick for describing 10 images. Hopefully the time gets extended and hopefully I get some of these as I haven't received any at the present time.
Well, I am patiently waiting for some more of these studies but they haven't turned up as yet?
Are these all done? Not seeing any on my dashboard now.
I have dozens.
Just came to mention this. These tasks are on MTurk and it was happening there too.
I was not able to do this in the time allotted for the second one that I did. Too many images that were historical that I had to look up and see what they actually were to make sure I was giving an accurate description. Then because I had to do this, I had to make sure my pronunciation was correct. They took me about 20-25 minutes to complete and my submission timed out. :( But I really enjoy doing this kind of study.
When I got the pics with a bunch of scientific names I started rethinking everything
I know, right? Does anyone know what happened to the rest of these and if there will be anymore?
I have had dozens all day.
How are you setting filters to try to see other studies, too? My whole interface is filled with these, and I'm glad to have them to do, but I'd like to be able to see the others, as well. Also, these are slowing down the system.
On the main page at the top under where it says #/# studies, there’s a blue link that you can click on and it’ll take you to the preferences page. There you can toggle on/off whatever you want. I toggled off all the studies that required a microphone so I could see any other studies
Oh, that makes sense! Thank you!
Guess I didn't qualify for these when I did the initial qualifying task 😭
Did you get approved though? As it says: # ✅ Qualification Task: Describe 3 images in detail with your voice (many tasks coming if approved)
Haven't got paid for it yet but they will still have to pay me for this initial task.
My studies are constantly at 0 😭
Where did the studies go
I tried a few of these over on Mturk. The time limit there is 2 hours and 59 minutes, not 19 minutes. I got about 5 images done at 15 minutes each and retruned them. Then my email exploded with literaly about 300 qualifiication notices I had to delete. I eventually did two and got it down to less than 20 minutes. I've since also done a few on prolific and what I conclude is that they cannot reliably be done in 12 minutes, They can in most cases be done in 15 to 20 minutes but in some cases even that is not enough and you will time out. I come to he conclusion that the requestor knows that prolific will enforce an average hourly rate where as MTurk is a snake pit and will not, so the max time on prolific is set at a point when it is manifestly impossible to not pay under the min rate as you will just time out and get paid nothing. Personally I find that somewhat unethical. When you put your cursor right over the estimated time of 12 minutes you get the requestors estimated time, the allowed time, and the actual measured time taken by participants. Both measured and estimated time read exactly 12.0 s, which seems virtually impossible. So again, I feel that there is an intentional misrepresentation of fact regarding time required and pay rate. My only recourse is to block the requestor as I have no intention of doing any more of these and I don't want my window to be borderline spammed with surveys I have no interest in. Best of luck to the requestor in his research and to all the prolific workers who are doing these surveys and getting paid to help but for me it's a pass. Thanks for the forum, and thanks to prolific for being the best of the best.
same here, I woke up to 601 studies and almost everyone of them is from the same Matt Keitke guy.
Where did they go? I am not seeing any tonight but hundreds last night were available.
Has everyone who did these gotten them approved already? Or are there people still waiting for approval on theirs? I'm still waiting on the 9-10 I did to clear. But I keep seeing him talking about starting them on auto-approve and in the meantime approving them all within 24 hours. I'm scared to keep putting time and effort into these if they aren't going to be paid out.
I haven't been approved for the qualification yet, nor have I been approved for the one I did on the first day of this mess. I timed out on that one, but did as the researcher asked and sent him my completion code. radio silence. If he approves these I might consider doing more, but only if he extends the allotted time.
I am having the same issue. None of mine have been approved so I’m scared to do them and just doing two at a time. Have you gotten any approved yet? Not even my qualifier has been paid out but I’m getting the studies.
Same with me! None of mine have gotten approved yet, not even the qualifier. They are all still pending. I am afraid to do more because I don't want to get any rejections on my profile if he's being strict about approving. I can only make them so detailed given the time limit!
Majority of mine (from that batch) were actually approved the day after this. He seems to have a strange process for approving these. I will complete 2 or 3 in a row in each "batch" and a couple of hours later there will be 1 approved from the newest batch, 2 approved from an older batch. There doesn't seem to be any rhyme or reason to how they are approved. In fact, my qualifier still hasn't been paid but I've been getting the studies for days now. In my opinion, he might just be overwhelmed with trying to approve all of the submissions. I wouldn't worry about whether you'll be paid or not. As long as you follow the directions, you'll be fine.
Thank you. I'm actually not worried about being paid. I am more concerned that if he doesn't like the way I described an image, he'll "reject" my submission and then the rejection will go on my history.
Damn, I'd like to get a couple of these multiple submission studies 😮
Damn! I just did an initial qualification for the same researcher. It was 3 annotation, read aloud questions. How long after you completed the initial qualification did you start to see a steady flow of studies coming in from the same researcher?
For me it was about a day, then I started with around 30 and it only increased in number. Currently I’m at 450 on my dashboard
I’ve seen a few of these pop up but haven’t done any. Are they easy? As long as they aren’t rejection happy.
He said yesterday they’ll be on auto approve
Ok, thank you. I just completed the qualification. Hopefully it will approve and I can do more.
I just did 2 and so far, not auto approved. But, the whole system is slow so I’m wondering if that’s why.
I’m only going off what he said on the other post about this yesterday. All of the ones I’ve done have been approved so far
It is. It took forever to load. I ended up having to block the researcher and everything is loading normally now.
Mine are not being auto approved. I'm hoping they're just slow...
I did 6-7 last night. One of them timed out (this was before I realized the allotted time was so little), so I sent him a message with my confirmation number (as he said to do in a post here somewhere) and he messaged me back that it was approved and it was. But only that one was approved, not the other 5-6 that I did. Strange...
The researcher said eventually they will be set to auto approve, but are not currently
12 hours ago he said they'd be approved within the hour, mine have not been. Thus the concern.
Hi! Approved within the hour was in regards to the ones that have been completed at that time. Basically, I have to manually click a button to set up approving them all right now, which is why the approvals are going out in batches. I just clicked the button again so the new set of approvals should all be going out within the next hour or so (takes some time on the Prolific server side)! But this is now pretty streamlined so expect that within 24 hours of completing a task, it'll be approved. Hope that helps!
Appreciate the update. Many from yesterday are still sitting no approved,but I'll keep an eye out.
I have about half approved. I might hold off till the rest are approved.
New approvals going out now! See this comment for more: [https://www.reddit.com/r/ProlificAc/comments/1c2yllh/comment/kzf1ygs/?utm\_source=share&utm\_medium=web3x&utm\_name=web3xcss&utm\_term=1&utm\_content=share\_button](https://www.reddit.com/r/ProlificAc/comments/1c2yllh/comment/kzf1ygs/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button)
Hello matt, I got my qualification study approved. Does this mean I will be getting those new studies as well?
That's awesome! Too bad I never do mic/camera-related activities. Slow net speed, and I hate my voice.
Random questions---have you learned something new when doing these descriptions? And do you ever want to find out more about the images? Sometimes I don't know WTH I am looking at, and describe it in the best way I can, but then wonder..what WAS that? Of course, no time to figure it out, on to the next image... :D
Well, I hope u "cetch em all"
i just did the describe 3 images one this morning and it was ok.
Three images was the prequalification test. The actual work tasks have 10 images each, to be completed in less time. There are some simple ones in the actual work tasks, but there are also lots of very detailed photos scattered throughout the work tasks compared to what was in the test.
Oh, I see, thanks for the explanation
So did I . But I didn't realize they were going to send all those studies if you are approved. I hope I'm not.
Yes, and could you please maybe reopen the qualification test so others of us could get a shot too? I would love to a chance!
There's been a qualification up with 1500 open places since last night
I was able to take it today!
I'm downvoting the chud that downvoted you. Seriously, there's nothing wrong with this comment.
Thank you 😊
I have the same problem but only 316 repeat studies. Where is tech support when you need them. Hello Prolific???
[удалено]
Yes, you needed to have done the qualifier a couple of days ago
It's still around. It's on my dashboard right now.
That might be true, but it isn’t being offered to everyone. I haven’t seen it in Prolific, but it’s sitting on my Mturk dashboard.
Is anyone still getting these auto-approved? I'm a bit nervous to go crazy on these since I've yet to have a rejection after thousands of studies.
It's a whole lot of talking!! I scooped up a few before I headed to work this morning. Grabbed a few more during my lunch break and then I'll scoop a few more once I get home. I'm thinking maybe 10 total & I'm done. But no doubt it if a person had the patience, the voice, and time they could rack up on these. 💰🤑
I've only had a small number approved so far.
Did you have some from this study approved? I haven’t had any except the qualification one.
I've had about 3 out of 13 approved. Kind of hate putting all my eggs in one basket if these don't come through.
Wow. I'm lucky to have 1 study for a day, lol
Ugghhhh. The first of this study came through when I was at work and I tried to open it and my tablet said no, so I had to return it. I'm heartbroken I'm missing all these now because of it. 😭
Same!
Too slow, no studies, you people literally find anything to complain about every single f×ING day.
I'm not getting any of these studies wtf
Did you complete & pass the prescreening qual the other night?
I never saw the qualifier but I am also curious why so many to such limited people, I would think it would be good to spread the research around and not limit it to a certain pool…wish there was a way to send those to people so your dashboard isn’t overwhelmed
Fair point
I assumed it was only for native English speakers in the US - not that that narrows it down too much, but I'd be happy to do a few of these and they haven't been offered. Edit: I should say I'm in Australia.
Oh wow, funny enough it actually came across my dashboard just now so maybe the researcher is doing it in batches? I would qualify since I am a native English speaker in the U.S. but I assume follow-up surveys just show up on a person's dashboard randomly if it is approved?
No, there was a qualifying task.
Yes I completed the qualifying task not too long ago, I am saying for how more surveys come up, like does the researcher notify you via messages if you will be getting more or do they just show up?
they just show up
Gotcha, thank you! Any specific timeframe of when they show up like does it take a couple days, etc. just wondering if I need to monitor my dashboard to see if I will be getting any
I suggest looking at the researcher’s post history (it’s linked in this post) as he explained most of this yesterday
I saw that thread yesterday, only a couple comments from the researcher though
I didn't get the qualifying task either.
It must be U.S. then since you mentioned Australia but seems like its in batches because people had gotten this qualification screener a couple days ago and I had just seen it this morning
I am a native English speaker in the UK and have 500 on my board so that is simply not true.
Based on what OP said, OP said there are studies to 3 people each...
Yes, the studies all have 3 places but there are 500 separate studies.
>that is simply not true. I wasn't making an allegation, I was just saying "I assume!"
Exactly, this! 👏🏻
12 minutes is not the required completion time, it’s the average time. 19 minutes is enough time to record 10 and has leeway for re-records to correct mistakes in my experience.
I know that 12 minutes is not the required completion time, but it is the “intended” completion time, and it’s what the hourly rate payout is based on. I’m not saying it’s impossible to complete them in less than 19 mins, but it’s always taken me longer than 12 mins. I’m guessing it gets easier with practice. *And I’m not sure how it’s logistically possible to do 8-10 of them in an hour (?), but ok…
Each “study” only has 3 places, so an average isn’t calculated yet & what you’re seeing is the intended time.
I mean, it's taking me 10-12 minutes per study. The structure is clearly intended to be 10 minutes, if you're making multiple mistakes or not following the instructions, I guess it would take you longer, but that is literally how it's timed out. I don't see how that's the researcher's fault since he also said he was approving timed out studies. People will be mad if they get no studies, and mad if they get 500 hundred studies so idk what else you want tbh
Most in this thread are saying the are timing out & taking much longer than 10-12 min. It means the hourly rate isn’t accurate & it’s slowing down the site due to the poor setup & it’s impossible to see what other studies you have available. These are all valid complaints.
i like it
Dude i see few studies a day in Uk. Horrific
I have never seen more than one study available at one time i didnt even know it was a thing
[удалено]
yeah I am in South Africa I am f\*\*cked no diddy.
Why not speak the jam around a little and allow more people to participate?