Others & Etc. / TOCHI 2013

A To-Do List Can Give You a First Step

TaskGenies tests whether action plans written by other people or reused from similar tasks help someone finish what they meant to do.

TaskGenies: Providing Action Plans Helps People Complete Tasks

Nicolas Kokkalis, Thomas Kohn, Johannes Huebner, Moontae Lee, Florian Schulze, Scott Klemmer

Overview

“Prepare for a trip” is easy to put on a list. It leaves the first action unresolved. TaskGenies asks whether people follow through more often when someone else supplies concrete steps. The researchers test plans written by paid crowd workers, contributed by a community, and adapted from earlier tasks.

Key Findings

Give a stalled task a first move

A to-do item can name a desired outcome without telling the person what to do next. TaskGenies lets a worker supply concrete steps, which the recipient can add as subtasks. The research follows two questions. Does a plan change what people finish? And how can a service provide plans without commissioning each one from scratch?

Figure 1. An action plan turns a high-level task into steps the recipient can use.
Figure 1. An action plan turns a high-level task into steps the recipient can use.

Teach a crowd what a usable plan looks like

The Genies workflow starts by asking workers to rate plans for other tasks. Each worker then writes two steps before the interface reveals suggestions from prior submissions. Delaying those suggestions gives people room to think independently, while still allowing a good step to make its way into later plans. The system accepted the first plan to clear an average rating threshold of 1.0 on a scale from −2 to +2, based on five ratings. It averaged 2.2 workers per accepted plan at that threshold. Workflow and acceptance rule

Figure 6. The crowd workflow separates rating and initial thinking from the moment when a worker sees and can adopt prior suggestions.
Figure 6. The crowd workflow separates rating and initial thinking from the moment when a worker sees and can adopt prior suggestions.

Track what people actually report finishing

The Crowd study separates receiving a plan from simply receiving a reminder to make one. Participants with crowd-written plans reported completing 47.1% of tasks within a week. That was 9.3 percentage points above the control group and 7.1 above the prompt group. The two comparisons were statistically significant after correction for multiple comparisons. The study did not detect a statistically significant difference between the prompt and control groups. Figure 2 shows the three completion rates with 95% confidence intervals. Study 1

Figure 2. One-week task completion was highest in the group that received crowd-written plans. Error bars show 95% confidence intervals.
Figure 2. One-week task completion was highest in the group that received crowd-written plans. Error bars show 95% confidence intervals.
StudyPlans suppliedComparison
Crowd47.1%37.8% control, 40.0% prompt
Community55.5%49.9% control
Recycle56.2%43.1% control

The trials use different participants and conditions. The comparisons belong within rows. Participants report whether they completed their tasks within a week, including tasks no observer could independently check.

Where getting the first step helped most

Making a plan was especially useful for tasks that had lingered or were described at a high level. The difference was also clearer when someone could understand the task without asking its owner for more context. The table reports selected task-level comparisons from Study 1. The counts denote tasks in each subgroup. Each subgroup comparison is separate, and the counts should not be added across rows. Study 1 subgroup results

Task subgroupCrowd plansControl
High-level44.3% of 341 tasks29.7% of 381 tasks
Lingering40.6% of 780 tasks30.5% of 820 tasks
Nonlingering53.1% of 780 tasks46.8% of 820 tasks
Sufficient context44.7% of 828 tasks36.1% of 893 tasks

Share the work without putting it all on the users

People who received plans in the Community study reported finishing more of their tasks than the control group. The 88 participants in the community condition created 655 plans and paid workers supplied the remainder. The authors report that community contributions reduced the paid workers’ load by 37%. Within the community condition, people who contributed more plans tended to finish fewer of their own tasks. This association leaves open whether writing plans took time away from their own tasks or whether people who contributed more differed in other ways. Study 2

Reuse a plan when the new task is genuinely similar

The Recycle study draws on about 6,000 previously planned tasks. GeNiLP resolves word senses with WordNet and compares tasks by matching word senses across the two descriptions. Its decision rule can be written compactly. With archive CC, incoming task tt, and similarity score sim⁡\operatorname{sim}, the system selects

s∗=arg⁡max⁡s∈Csim⁡(t,s).s^\ast=\arg\max_{s\in C}\operatorname{sim}(t,s).

It reuses the plan for s∗s^\ast when the score exceeds 0.30.3 and commissions a crowd-written plan otherwise. The authors chose the cutoff empirically for this study. The system reused plans for 95% of tasks, and the reported Recycle comparison refers to that subset. A shared verb can produce the wrong match. Calling someone about a broken car can be scored as similar to fixing a water heater. Study 3 · Similarity algorithm

Judge the plans separately from the tasks

The fourth study compared Genies with parallel writing, serial revision, and writing followed by revision. It used ten tasks and 30 plans per workflow, scored on a ten-point scale by a rater unaware of the conditions. Genies received the highest average rating, with statistically significant pairwise differences against each alternative after correction. This experiment measured perceived plan quality. The first three experiments measured self-reported task completion. Study 4

Figure 12. The Genies workflow received the highest average action-plan rating among the four workflows in Study 4.
Figure 12. The Genies workflow received the highest average action-plan rating among the four workflows in Study 4.

Questions & Answers

The opening answers are simulated author responses based on the paper.

Q: Why reuse a plan?

A: Some tasks are close enough that the same structure is useful. Reuse saves writing effort. It needs a good match and can fail when a shared word disguises a different goal.

Q: Are other people’s plans always suitable?

A: A plan can miss a deadline, a personal constraint, or a resource the task owner lacks. The paper finds benefits in the tested groups and discusses the role of context.

Q: What does comparing a supplied plan with a planning prompt tell us?

A: A reminder to plan can itself prompt someone to get started. The Crowd study includes a group prompted to write its own plan, which lets the researchers compare that reminder with receiving a ready-made plan. The Recycle study had 39 participants and used self-reported completion. In the Community study, participants who created more plans for others tended to finish fewer tasks of their own. That association cannot tell us whether helping others caused the difference.

Future Ideas

Ask for the one detail the old plan needs

For a volunteer project board, tag tasks with a goal and constraints such as location or equipment. Show a reusable first step when those tags match. If a missing detail could change the action, ask the owner for it before proposing a plan. Compare this approach with always reusing, fresh human planning, and a plain checklist. Track independently confirmed progress where available and the time people spend answering the extra question and correcting suggestions.

Include the time it takes to elicit the missing detail. Where possible, confirm completed work independently and distinguish using a suggested plan from simply receiving one.

Sources