1. Q: What’s the motivation for the CRC?
A: We’re exploring new models for exchange of information and engagement with political issues. We don’t claim the CRC model is the answer: it’s one new metaphor and mode of collecting information. Will the CRC encourage people to think about public issues and how public opinion gets represented? We’re curious to see how people respond, how patterns of responses change over time, and if the grading will be a catalyst for the discussion, where participants propose new issues for the next report card and evaluate the suggestions of other participants. We’ll keep working to develop new versions and models based on the responses and data.
2. Q: How can I learn more about the six issues we are grading?
A: To learn more about the six issues, please visit californiareportcard.org/issues/.
3. Q: How can I grade K-12 Education when there are so many aspects, the effectiveness of the teachers, or the quality of school facilities, etc.?
A: That’s a great point; we’re asking people to give an overall grade, which is indeed a coarse measure! But it’s also like grading a student for a class, they may have done well on some things, poorly on others, etc…
4. Q: After I enter each grade, the Median Grade based on input from all previous participants is revealed. Then I can change my grade: Won’t this bias me and other participants?
A: Revealing the Median values is intended to provide instant feedback in contrast to traditional polls and surveys. We realize this may have a biasing effect: some may adjust their grade to be closer to the average resulting in a form of regression toward the mean. We believe this will be rare, especially for Californians 😉 . We will include details on the number of such adjustments and the before and after values in the open dataset.
5. Q: How are the spheres arranged on the table?
A: The placement of each participant’s spheres is based on how he or she graded the first 6 issues. Think of each issue as one dimension of a 6-dimensional space. The grades you assign define a point in this space. We use dimensionality reduction (currently Principal Component Analysis) to project that space onto the table. If you and another participant assigned very similar grades, your spheres will be close together. Spheres that are distant from yours belong to participants who graded very differently than you.
6. Q: Why do I only see 8-10 spheres arranged on the table?
A: To avoid overcrowding, we display only a few spheres at a time for grading. The selection of which participants’ spheres to display is random but biased toward newer spheres, those that have received fewer ratings. Specifically, the likelihood of displaying a sphere is based on its standard error = stdev / sqrt(k) where k is the number of times it has been graded so far, and stdev is the variance of those grades. Over time, this ensures that the confidence, or error-bar on the estimated grade for each sphere, tends to be the same length for all spheres. Each time you finish grading a set of spheres, the table will be refreshed with more spheres.
7. Q: Is there a meaning to the different sizes of the spheres?
A: Not yet. We will use those variables in future versions, for example to reflect the score of each participant and the date he or she joined the discussion.
8. Q: What will you do with the grades you collect?
A: The grading data will be updated daily and available on a public statistics page that will also display the median grades geographical (by zip code). We will also post the full anonymized dataset online for anyone to study after 20 March 2014.
9. Q: Why are there so few grades on the issue I suggested?
A: Your grading stats are updated once a day (at midnight). Your suggestion will be displayed as a sphere and graded as new participants join the discussion. It can take up to a week for enough participants to review and grade your suggestion.
10. Q: When will version 2.0 of the Report Card be available?
A: We concluded data collection for version 1.0 in early March and presented the data on 20 March 2014 at UC Berkeley (http://californiareportcard.org/data/). We will incorporate the six best suggestions for issues to include in version 2.0 and plan to release it as early as Spring 2014.
11. Q: How can I learn more about this project, its history, related projects, and the team that built it?
A: Please visit The California Report Card website for more information.
12. Q: How can I contact the team about a problem or question?
A: To contact the team, please visit californiareportcard.org/contact/
13. Q: Why doesn’t the CRC respond as fast as my other phone apps?
A: Technically the CRC isn’t an app, you don’t have to download and install it. It’s a website that uses the HTML5 language for compatibility with most desktop, laptop, notepad, and smartphone browsers.
14. Q: When I turned my phone sideways the report card stopped working.
A: Apologies! We know about this bug. It’s very hard to accommodate vertical and horizontal screens using HTML5: we’re working on it…
15. Q: Why do the the sliders require dragging? If I tap on A, it should go there.
A: We decided to remove the tap feature since as you slide down to see more issues you may accidentally tap the sliders.
16. Q: Is the CRC funded by a company or special interest group?
A: No. The CRC is an academic project supported by the CITRIS Data and Democracy Initiative.
17.Q: Is there a way to directly discuss and debate issues with other participants?
A: Not yet, but we will set that up if there’s interest. Please email Brandie Nonnecke at firstname.lastname@example.org if you’d like to join an online discussion forum.
Please send us an email to suggest new Q (or versions of Answers!) for this list. Emails can be sent to email@example.com.