What people think about FT

We ask FT participants for feedback at a couple of points in the process; first when they are notified whether they got time, and again a few months later when they are hopefully writing up their excellent results. The initial survey contains a mixture of multiple choice questions, and boxes in which people can give their own answers, comments, and suggestions. Here’s a snapshot of the results from that survey so far:

feedback

 

Most people have reviewed telescope proposals before, and a similar number find the reviewing process about as easy or difficult as they had expected. The words most frequently used to describe the process are “interesting” and “educational”. Admittedly, this is probably prompted by the fact that we ask “How did you find the reviewing process? E.g. interesting, challenging, educational, time-consuming”, but people do seem to agree that they benefit from reviewing others’ proposals.

We also ask how the quality of the FT proposals compares to those they have reviewed in other contexts. The large majority of participants rate the FT proposals as similar to others they have assessed, with a couple even finding the FT proposals of higher quality. One or two reservations were expressed in the comments. For example:

“Quality was comparable to HST proposals for the most part, though there were clearly some (~30%) that were hastily put together.”

Users of the FT program receive all the reviews written about their proposals, and we have always hoped that this feedback would be helpful. This does seem to be the case, with >75% of users reporting that the comments on their proposals were “mostly helpful”. As one user commented:

“The practice of providing each proposer with the full set of reviewer reports (rather than a diluted summary) was unusual, and I welcome the change. This provides much deeper feedback.”

The respondents aren’t very bothered about the possibility of the peer review system being abused. Only one person reported being “moderately concerned”. Of course, this is a biased sample; if you’re worried about the review system you’re probably not going to submit a proposal in the first place. Nonetheless, it is gratifying that, having been through the whole process, these people generally saw no major cause for concern. Almost everyone said they’d be “quite likely” or “very likely” to use the FT program in the future.

The feedback surveys have also given us a number of suggestions for improvement. Some of these (e.g. expanding to Gemini South, doing away with separate FT observing nights) have already been implemented, some (e.g. shortening the proposal form vs not shortening it) don’t agree with each other, and others (e.g. “the final decision tree is not visible to the reviewers”) are on our to-do list. If you use the program in the future, please be aware that we do appreciate these surveys and we are paying attention to the results.

Posted in Uncategorized | Comments Off on What people think about FT

November demand, first GS Fast Turnaround observations

The number of FT proposals received at the end of November dropped slightly compared to September and October, from 16 to 12.  As in October, the majority of proposals were for Gemini-N, with the three Gemini-S proposals requesting 7.5 hours between them (out of up to 20 hrs available). If you’re considering submitting an FT proposal for an instrument at Gemini South, this might be a very good time to do so…

oversub_vs_T

Above: the blue bars and left-hand axis show the number of proposals received, while the squares and right-hand axis show the number of hours requested (red) and awarded (green). The dashed line denotes the 20 hrs/month/telescope available for FT programs.

The PIs of the first successful Gemini South FT proposals were notified on November 20th. The first data were taken 8 days later, and the top-ranked program is about 75% complete at the time of writing.

The December call for FT proposals has just been published. The 16A instrument schedules are not yet available, but we’ll add that information to the call as soon as we have it.

Posted in Uncategorized | Comments Off on November demand, first GS Fast Turnaround observations

Your monthly FT demand report, and more

FT demand at the October deadline was healthy again, with 16 proposals received (see the figure below – the blue bars and left-hand axis show the number of proposals, while the squares and right-hand axis show the number of hours requested (red) and awarded (green). The dashed line denotes the 20 hrs/month available for FT programs).

This was the first deadline at which Gemini South time could be requested through the FT program, and we received 4 GS proposals: one for Flamingos-2 and three for GMOS-S. These proposals request a total of 16.5 hours between them, and we can allocate up to 20 hrs per month per telescope. Assuming they reach the minimum score of 2.0, no technical problems are found, and they don’t all compete with each other in terms of RA, observing conditions etc., it is likely that all three will be awarded time. The review deadline is today (it’s always on the 14th of the month).

oversub_vs_T

Also, the first paper to include FT data was recently published. The paper, “Time Delay Measurements for the Cluster-lensed Sextuple Quasar SDSS J2222+2745”
by H. Dahle et al., was featured in this Gemini web article. If you have a result based on FT data that you think would be a good subject for a web feature, press release, or other publicity, please get in touch and maybe we can help each other out.

 

Posted in Uncategorized | Comments Off on Your monthly FT demand report, and more

Gemini South now accepting Fast Turnaround proposals

For the upcoming October 31 deadline and beyond, Fast Turnaround proposals can request time on Gemini South. GMOS-S and Flamingos-2 are available, along with a limited amount of time on GPI. We aren’t offering GeMS at the moment, but it would be useful for us to know if GeMS+FT might be useful for you in the future. And if you’re applying for time with GMOS, please let us know in your proposal whether your observations could be done with either GMOS-N or GMOS-S. See the Call for Proposals for more information.

We plan to operate a single review process, so reviewers in future cycles can expect to be assigned proposals for either telescope. We will allocate up to 20 hours per telescope each month without trying to balance the time between telescopes. We’re very much looking forward to your Gemini South proposals, and to opening up the FT program to some new users.

Posted in Uncategorized | Comments Off on Gemini South now accepting Fast Turnaround proposals

FT demand is back…

So, you people really know how to surprise us… After several months of unexceptional demand for FT time, the last thing we were expecting was a near-record number of proposals in September. With Gemini staff now being able to submit proposals, and FT starting up at Gemini South next month, we thought that demand was likely to pick up a bit soon. However, we expected you all to be too busy writing regular proposals for the 16A deadline to submit many FT applications.

In the end, though, we received 16 proposals at the end of September. Even accounting for the 3 staff submissions, this is our second-highest number since the program began. The figure below illustrates this – the blue bars and left-hand axis show the number of proposals, while the squares and right-hand axis show the number of hours requested (red) and awarded (green). The dashed line denotes the 20 hrs/month available for FT programs.

oversub_vs_T

 

You’ll notice that two cycles are annotated with “cycle cancelled”, while one says “Used external reviewers”. This marks the policy change detailed in this blog post, where we used to require >7 proposals in order for the program to run, but now we add extra reviewers as needed.

Good luck to this month’s proposers; we’re as curious as you to see the outcome of this month’s FT cycle!

 

Posted in Uncategorized | Comments Off on FT demand is back…

New FT mailing list

Just a quick note to say that we now have a mailing list that will be used to send monthly reminders and inform users of changes to the program that may affect or interest them. To sign up, send a message to “Gemini-FT-reminders+subscribe” at gemini dot edu.

Posted in Uncategorized | Comments Off on New FT mailing list

New FT opportunities

We’ve just received approval from the Board to start accepting Fast Turnaround proposals for Gemini South instrumentation. We’re still finalising the implementation (such as which instruments/modes can realistically be supported), but the first call for proposals including Gemini South is probably going to come out in October. The plan is that proposals for both telescopes will go into a single pool with time allocated according to merit rather than enforcing a strict 50:50 North:South division.

Also, Gemini staff can now be PIs on FT proposals (this was prohibited during the first few months while the scheme was becoming established). Special restrictions will apply to those of us involved in running the program. More staff are trained in FT support than are needed each month, so from now on we will work in pre-determined shifts and only be permitted to submit proposals in our off-duty months.

 

Posted in Uncategorized | Comments Off on New FT opportunities

MOS observations now available through FT

Now that we have some experience running the FT program, we believe we can support multi-object spectroscopy with GMOS North. We welcome MOS proposals for the August 2015 deadline and onwards. This is subject to a few caveats:

  1. We will only be accepting programs that don’t require pre-imaging.
  2. All our MOS masks are cut in Chile and have to be Fedex-ed to Hawai’i. This slows things down by 2-3 weeks, meaning that MOS observations will have somewhat less chance of getting executed in the 3-month period for which FT programs are valid. [January 21 2016 edit: depending on the status of the mask cutting machines, sometimes masks are cut in Hawai’i and shipped to Chile. Either way, delays can be expected.]
  3. If you would like to re-use a mask that was cut for a previous queue program, please let us know in advance. Masks are not kept indefinitely, so your old mask may no longer be available.
  4. Due to staffing limitations, mask designs are only checked on Fridays. PIs of accepted MOS programs will need to time their mask-making around this.

We look forward to receiving your MOS proposals. If you have questions, please contact the FT team at “Fast dot Turnaround at gemini dot edu”.

 

Posted in Uncategorized | Comments Off on MOS observations now available through FT

How this works, and how to succeed

Your correspondent has been meaning to write a post explaining a little of what goes on behind the scenes in the FT program, showing what participants see as they go through the process, and giving a couple of tips for writing successful FT proposals. Here, finally, is that post.

1) The process

Our proposal deadlines are at midnight HST on the last day of each month, and at this point the FT software gathers the proposals from the server and creates a web page showing what has been received. In the morning, members of the FT support team check that each proposal meets the criteria to be a valid FT proposal: that its PI is from a participating partner, the target RAs are within the acceptable range, etc. We also manually check for conflicts with current queue programs, which wouldn’t have been found by the archive search carried out by the Phase I Tool before the proposal was submitted. We check a button to include or exclude each proposal (as shown by the screenshot below), and then click a button to generate The Matrix…

include_exclude matrix

The Matrix shows us which proposals have been assigned to each reviewer. The software matches proposals to reviewers based on the keywords in the proposals, as far as possible. Those are the numbers shown in the cells in the Matrix. The software also looks for obvious (potential) conflicts, making sure that reviewers aren’t assigned proposals on which they are PI or co-I (red cells) and flagging proposals with targets in common (yellow cells). By clicking on the cells in the Matrix the FT team can select, deselect, and veto any of the assignments. This allows us to ensure that, say, reviewers aren’t assigned proposals for very similar observations of the same non-sidereal object, which wouldn’t always be caught by the automatic target comparisons. In general, though, we interfere as little as possible in this step.

Once each reviewer has been assigned their proposals (up to 8 each), we simply tell the software to send out standard emails to each reviewer giving them instructions about how to proceed. This usually happens within about 12 hours of the deadline. When a reviewer clicks on the link in their email, they are first asked to set a password. The next step is to agree that they will behave ethically and follow the rules of the program:

ethical_participation

Having agreed to these terms, the reviewer is then shown the list of proposals that have been selected for them – title, investigator list, and abstract – and asked to declare whether or not they are able to provide an unbiased review of each one. When a proposal is declined, a replacement is offered (when sufficient proposals have been received).

assignments

Only at this stage do the reviewers proceed to the review form and gain access to the full proposals. The review form presents the assessment criteria and requests a numerical score and brief written review of each proposal. It also asks the reviewer to state their own view of their knowledge of the subject area of each proposal on a scale of 0-2. This is currently just one of the tools we use to evaluate the program; it is not used to weight the reviewers’ scores.

how_to_review reviews2

The reviews must be completed by the 14th of the month, at which point access to the forms is removed and the team is notified that the review cycle has closed. The Matrix page now shows us a list of proposals ranked by their mean score, with various tabs in which we can see individual reviews and other information. We figure out (currently using our brains and a whiteboard) which programs are likely to fit in the available time, always trying to stick as closely as possible to the reviewers’ ranking. Any proposal must obtain a mean score of at least 2 (on a 0-4 scale) to be considered, no matter how much time is available.

Over the next couple of days we check that the provisionally-accepted programs are technically feasible. If any are not, we re-examine the list and replace the problem proposals as appropriate. Once we have the final list, we mark each proposal as accepted or rejected, upload our technical assessments, and tell the software to notify the directorate of the selection. While we await their formal approval, we manually create skeleton programs in the Observing Tool. By the 21st of the month everything is in place to generate the emails that notify the PIs and reviewers of the outcome of the proposal cycle. Everyone receives their mean score, all the individual reviews of their proposals, and the technical assessment if one was done.

The PI immediately has access to their program and can begin to set up their observations.  Starting in 2015B we will be merging FT programs into the queue rather than using separate FT nights. FT observations will then be available for scheduling as soon as they have been prepared, which is hopefully an incentive to set them up ASAP…

2) How to write a successful FT proposal

This really comes down to just a couple of things:

  • First, make sure your science is accessible to a broad audience. Although the keyword-matching algorithm tries to match reviewers to proposals in similar or overlapping areas, the pool of potential reviews is small enough that your idea is going to be judged by people outside your field. It’s really helpful if the quasar person can understand why your brown dwarf observation is worth doing.
  • Second, make the case for why your proposal is a good candidate for the FT program. We at Gemini believe that FT time is for any kind of good science, regardless of whether the target is fading or about to disappear behind the sun. This is clearly stated on the FT web pages and review form, and we now include this in our initial instructions to reviewers as well. However, we have observed a tendency for reviewers to give some weight to the “urgency” of a program and its need for rapid response. If you simply have a good idea that you want to pursue right now, then we’d encourage you to explain in the proposal that this is super exciting and you’ll get on the data right away.

3) One final thing

We received 7 proposals at the July deadline and the cycle is running as usual. We now eagerly await your August proposals.

Posted in Uncategorized | Comments Off on How this works, and how to succeed

Change to review process when <7 proposals received

As noted in previous posts, the FT program requires a minimum of 7 proposals in any month for the program to run during that month. We have now had 2 proposal deadlines at which that threshold wasn’t met, leading to the cycle being cancelled and probably to some disappointment on the part of the PIs. To avoid this happening again, we are in the process of convening a standing panel of “external” reviewers. In future months when we receive <7 proposals, the submitters will still review each other’s proposals but we will also request extra reviews from this standing panel.

Please note that this only applies to months in which proposals are sparse. When we receive 7 or more proposals, the peer review process will continue as usual.

 

 

Posted in Uncategorized | Comments Off on Change to review process when <7 proposals received