Frank Answers Fridays: August 8, 2014

Blog Date: 
Friday, August 8, 2014 - 14:12

Today’s good question comes from Aaron Schmitz, an alumnus from FRC Team 2502, Talon Robotics, from Eden Prairie, Minnesota, USA:

Question

Hello Frank & co.,

Allow me to introduce myself by saying I’m alumni of FRC 2502 who works for a well-known Fortune 50 software company. Acting as de facto liaison/volunteer wrangler between this company and FIRST’s regional partner, I can say with confidence that there are many outstanding developers here and at other technology corporations who are happy to donate their time certainly as volunteers and mentors but in other ways as well.

Based on your humility on this blog, I’m sure you’ll be the first to admit that at the beginning of the season the scoring system was a source of frustration to many. As a referee during weeks one through five I was able to directly observe the evolution of the FMS over the competition season, and certainly by week three most of the issues were resolved but that’s a tough break for teams whose only event was week one or two.

That brings me to my question: how does FIRST plan to mitigate issues with the scoring system so that in future seasons behavior is rock solid week one or *crosses fingers* week zero, and more specifically either for 2015 or down the road does FIRST have any plans to continue it’s well received trend of increased transparency in order to achieve this result. There are thousands of industry professionals and thousands more students who would be happy to audit (or even write) code at no cost to FIRST without compromising the integrity of the game – perhaps freeing up your developers to work on cool things the community is asking for (tablet app for the inspectors?). Frank, tell us how we can help!

Best regards,

Aaron Schmitz

Answer

Aaron, thanks for the question. You are right that the scoring system was frustrating for many the first few weeks of the 2014 season. We were frustrated with it as well. It did not have the level of quality we had worked and planned for. The Aerial Assist scoring rules were among the most complex we have had to deal with recently, with many points and activities depending on sequences of events, rather than a simple ‘put object X in opening Y and get Z points’. We discovered several corner cases, unlikely events but still possible, that did not behave as they should have. Also, development of the game ran late, as this very different FRC game required much more than the usual effort to complete. However, regardless of the complexity of the rules or the lateness of the game development, we did not deliver the highest quality product possible, and for that we are sorry.

We have three specific approaches we’re taking for the 2015 season to help correct this. First, we’ve hired additional engineers to make sure we have the technical resources we need to do the job in the time we have. You may be familiar with our two most recent hires – Alex Herreid and Danny Diaz. In addition, we are taking a more structured approach to the game design process itself that should result in the game being completed earlier than it was last year, allowing us the time we need to put the system through its full paces, corner cases and all. Finally, the game design committee has determined that having ‘simple and objective rules’ is one of their top criteria for helping make decisions related to the 2015 game.* This should help reduce the complexity of the system required to support it.

For us to be able to test the scoring system/Field Management System (FMS), we need the full system, with all components – the FMS server cabinet and associated components (known as the Scorpion, after the manufacturer of the cabinet, not because it can sting), station control cabinets (SCCs), estops, team numbers, touch screens, indicator lights, etc, etc, etc.

Here are a few photos of parts of our setup in our Mission Control on the 5th floor:

(“Hey, why are those numbers showing, it must be a game hint!”  Listen, please – it’s not a game hint. Our game hints typically start with something like “This is a game hint!”)

Our software partners in developing the FMS for the last few years, 4FX, have a duplicate system, which allows them to test the system as well.

I appreciate your offer to get community support in developing our software. I’m sure that if we asked we’d get a number of people lining up to help. However, with the time pressure we are under, and the need to have immediate access to the actual equipment to fully test the software, I think it’s best for us to rely on our internal and contracted resources. I believe that with the changes we are making for the 2015 season, we will see significant improvements.  

Frank

* If you are interested, it ranks 4th out of 23 total game criteria, behind ‘Safe’, ‘Fun and Exciting’, and ‘Fair Penalties’. The 24th ranked criterion, ‘More Cowbell’ did not make the final cut.       

Comments

Would you mind sharing the other 19 game design criteria? Many of the FIRST community, myself included, would be interested to know what other criteria shapes game development.

Hi Nick. I'd be concerned folks would read too much into the other criteria if I released the full list. Also, the criteria typically shift from year to year, depending on the process we use, and could potentially include details that would give too much away before Kickoff.

May you post the dates of the 2015 Alamo Regionals please ?

Sure. We just received a fully executed contract last week for Alamo, we will be posting the dates soon. As a rule of thumb, we like to see fully executed contracts before posting dates, as a fully executed contract legally binds both parties, and we want to be very certain the events will occur on the dates and at the places we post. We know teams make their plans based on the information we provide.

Frank, If we are able to get the game design done earlier, does that mean that you will be able to provide the drawings of the new field to the kickoff field builders earlier too? We have a full sized wooden field course available that we use for many local demonstrations (week zero for our local teams to practice on before the ship date as well as several off season demos). It is difficult for our mentors to build the correct course pieces prior to kickoff since we do not know what the final assembly will be. We may be able to reuse pieces from an older year. We've over-spent for material

I understand your need to limit these responses, but it's difficult to completely convey my thoughts with these limitations.

Our early build team constantly spends over $300 for unused, unneeded lumber supplies in order to supply our local teams at the pre-ship event with a usable field. (Let's not even think about the wasted time of the mentors). We've been at the setup of our kickoff event until midnight fixing (previous constructed) things because of the lack of an early understanding of how the full course goes together.

The 2009 "leak" is so far behind us now, give us a break!

Hi Bruce. From a security standpoint, I think of the game as being in a windowless room. The more folks that have access to parts of the game before kickoff, the more doors we have to that room. We can work hard to make sure every door stays locked all the time, but as we grow, the number of doors grows, and it becomes increasingly difficult to keep up. We actually did have leaks in 2013 and in 2014. Both were minor, accidental, and happened just before kickoff, but I can't rule out the possibility that a big leak would mean even more restrictions. We're trying to strike the right balance.

I understand your need to limit these responses, but it's difficult to completely convey my thoughts with these limitations.

Our early build team constantly spends over $300 for unused, unneeded lumber supplies in order to supply our local teams at the pre-ship event with a usable field. (Let's not even think about the wasted time of the mentors). We've been at the setup of our kickoff event until midnight fixing (previous constructed) things because of the lack of an early understanding of how the full course goes together.

The 2009 "leak" is so far behind us now, cut us some slack!

Add new comment

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.

All comments should embody the FIRST values of Gracious Professionalism® and will be moderated prior to posting. Thank you for helping to keep the conversation civil and productive.