Greg's Front Office Football 2004 Extractor Suite

This suite contains programs to dump players, scouts, or coaches, plus a program to analyze draft files. The complete list of utilities is:

Extractor: The original. FOF Extractor process player report windows to dump a CSV file listing details about a group of players, and can also dump scouts and coaches. The key thing about this utility is that it uses an OCR algorithm to grab data straight from the screens, which means it can be a bit flaky to run (activating ClearType in Windows gives it fits, for example) but also means it can grab data not available in any other way. Notably it gives you scout-adjusted ratings for the player.

DraftAnalyzer: Extractor's sibling, this tool runs against an Extractor dump of a draft class to allow easy analysis of the draftees. It tells you who has good combine scores, who has poor combine scores, and allows you to adjust various weights to set your own rankings.

LogFileProcessor: A recent addition, this converts game log files to CSV for help in analyzing how your gameplan is working. Can also generate an HTML table of gameplan data from a given team's point-of-view.

Interrogator: Another recent addition, this utility reads the binary game data directly to dump CSV files of pretty much every internal game table. It does not dump anything that the game UI doesn't give you access to (so no actual player ratings for cheating), but its key purpose is to dump all this data in one place for easier sifting in Excel or to use in creating databases for league web sites.

GameViewer: An off-line viewer for the game log files that allows you to step through each play to see what happened.

Extender: Provides some spreadsheet views of your saved games / multiplayer leagues that aren't available in the base game. For example, it includes tables of scouts and coaches that include all of their attributes for easier browsing when trying to decide who to hire.

Installation Notes

To install, download the zip file below and then unzip into any directory. It will extract a Windows Installer file. Double-click the .msi file to install the suite.

The file included in the zip file above is a Windows Installer database file. If this does not install for you (if nothing happens when you double-click the .msi file, or you get an error message), you may not have a current version of Windows Installer on your machine. You can get a version for Win9x here, or for NT/2000 here.

Several of the utitilities (notably Interrogator and LogFileProcessor) require version 2.0 of the .NET runtime. You can get it here.

Source code for an older version of Extractor and DraftAnalyzer is available here.

Known Issues

• The current version of Extractor relies on bit-perfect comparisons of the digits and symbols it checks for. This means it will not work if ClearType (which anti-aliases the fonts and blurs their outlines) is active on the system. To combat this, Extractor will turn off ClearType while running and turn it back on once you quit Extractor.
• The Extractor utility has been tested with the standard Arial system font. Other default font settings (for example, some Windows Themes use Tahoma) can cause problems. Not all of the other fonts work well with the OCR algorithm.

RSS

That's right, I've got an RSS feed! Keep automatically up-to-date by putting "http://www.gstelmack.com/rss.xml" into your aggregator and you'll be automatically updated whenever I release a new version.

Version History

4.3.5 (October 17, 2006): (download)

• Draft Analyzer no longer closes when you hit "RETURN". It will instead attempt to "Find" the player entered in the Find Player text box (which is usually what you want, since you usually hit "RETURN" just after typing a player's name in this box).
• Draft Analyzer will now display an error dialog if it can't match one or more drafted players when you paste a set of drafted player notices from FOF and attempt to mark them as drafted. No more having people think someone is available when they were drafted 50 picks earlier but missed by Analyzer...

4.3.4 (October 16, 2006): (download)

• Fixed several issues related to onside kicks and any kickoff out-of-bounds, especially unhandled exceptions on kicks out of bounds and properly detecting the change of possession and drive start issues.
• Added the ability to "Mark" a player in Draft Analyzer. This just adds an asterisk on the player's row, allowing you to highlight and remember key players you want to keep an eye on.

4.3.3 (August 30, 2006): (download)

• Extender will now skip players with a corrupted draft year. Fixes a potential crash.
• Analyzer will now attempt to predict how well a player would change to another position. It will display one of the following codes next to each legal position switch:
  • '?' - predicted at less than 90% of current ability.
  • '-' - predicted at 90 - 99% of current ability.
  • '=' - predicted to remain unchanged.
  • '+' - may increase ability.
  This is based off a sweep of a large sample of rookies (~4000), and is data-driven, so that if an error is discovered in either the data table or in the method used to extract this data (I didn't write that utility, so please don't ask for it, but it was just an automated way of opening each player, clicking the "change position" button, and parsing the results), we can regen or fix it and Analyzer will just use the new data. And as always, this is a rough scout guess, so the actual switch may differ. This has not been run through an actual draft cycle yet, so its usefulness remains to be seen.

  This data is in the "RookiePositionWeights.csv" file that installs to the Extractor Suite directory, so you should just be able to open this straight in Excel if you are curious or have seen the very old much smaller spreadsheet that is on my web site.

4.3.2 (August 20, 2006): (download)

• Fixed up a few display fields (sacks, punts-in-20) in the Extender Draft view.
• Highlighted players in the Extender Draft view that have had solid careers. The algorithm used is a rough fantasy scoring that can be tweaked as needed (right now it seems a bit harsh on fullbacks, for example).

4.3.1 (July 22, 2006): (download)

• Moved some Analyzer buttons to menu items to clear up some UI room.
• Added a "Find Player" option to Analyzer to select a player based on their name. This will do a search through all players, looking for the given string anywhere in their name. This search is case-sensitive. Subsequent searches start from the same point, so you can keep clicking "Find Player" to step through all players that match the string.
• Fixed extract of college name "LIU/C.W. Post". Extractor was pulling "L1U/C.W. Post".

4.3.0 (June 18, 2006): (download)

• Went through the list in colleges.txt and added fixes to Extractor for all colleges that looked like they might have trouble with the OCR algorithm. Found an issue with "Northern 1owa" and just decided to scan the list for any other troublemakers.
• Added Index fields to the TeamData and CityData output files from Intterogator.
• Added a Draft screen to Extender so you can review old drafts and how the players from those drafts did.
• Tweaked Draft Analyzer default combine weights. Now 2.5 for all positions.
• Cleaned up more penalties and some other play lines in the Game Viewer.
• Game Viewer handles "St. Louis" much better now.
• Game Viewer handles blocked punts correctly.

4.2.6 (May 31, 2006): (download)

• Extender now lets you choose whether or not to show retired staff members.
• Fixed Extractor issue introduced in 4.2.5 that caused most players to have "NW Missouri St." as their college.
• LogFileProcessor now correctly identifies Goal-Line formations for the Gameplay output.
• LogFileProcessor now spits out Gameplay output whenever just one team is selected in the team filter. You do not have to also check the "Combined" checkbox.

4.2.5 (May 22, 2006): (download)

• Fixed extraction of "NW Missouri St." college.
• GameViewer now credits Defensed passes against the QB (the "Def" column will no longer always be zero).
• Fixed a major problem with how LogFileProcessor was assigning plays to offensive / defensive formations and playcalls when generating the gameplan summary file. Should be much more accurate now.

4.2.4 (May 11, 2006): (download)

• Extractor now turns off ClearType™ font smoothing if active and reactivates it when the program exits. You no longer need to do this manually.

4.2.3 (May 8, 2006): (download)

• The documentation is now distributed as a compiled help file.
• The GameSelectorControl used by LogFileProcessor and GameViewer now has a "Show Exhibition Games" checkbox that if unchecked will remove exhibition games from the list.
• LogFileProcessor will no longer crash on a missing log file.
• Extractor now warns the user if it detects ClearType™ font smoothing active.

4.2.2 (April 16, 2006): (download)

• LogFileProcessor now has a "Combine" checkbox that if checked causes all game output to go to a single "combined.csv" file. This is useful for people who want to dump a range of logs for one team only and then load it up in Excel to see how their gameplanning is working.
• If you check the "Combine" textbox in LogFileProcessor AND have only one team selected when you do, it will also write a "teamabbrev_gameplan.html" file to the output directory with some useful gameplanning stats.
• Fixed a bug in how FOFViewer assigned forced fumble credit (it would pick up the last tackler who also had an assist instead of just the last tackler) which could also cause a crash if a fumble occurred before anyone got an assist on a tackle.
• FOFViewer handles blocked kicks better.
• FOFViewer handles onside kick recoveries better.
• Some suggested layout tweaks to FOFViewer. Comments remain welcome.

4.2.1 (April 6, 2006): (download)

• Extractor will no longer write out jersey numbers when extracting the draft pool to avoid messing up Analyzer.
• LogFileProcessor and GameViewer now share the exact same UI code for selecting their game(s). In addition to this giving LogFileProcessor the much faster game log searching code that GameViewer has (so it initializes quicker), both now have access to filtering options to trim down the list of available games. This also implies that LogFileProcessor will only search for game logs under the FOF installation directory and associated with a saved game.
• LogFileProcessor now lets you set the output directory for the CSV files.

4.2.0 (March 29, 2006):

• New utility! FOFExtender has some spreadsheet displays that show data not available through the normal game UI, or organizes some of the same data in a different format. Suggestions for new presentations that don't involve cheating (showing real player stats, for example) are welcome.
• Fixed some college names that Extractor gets wrong.
• Extractor will now write out the jersey number for players if available.
• Extractor now has a checkbox that allows you to enable or disable the extracting of contract information (which can slow down the process unnecessarily if you don't want contract data).
• Corrected Staff Role lookup in Interrogator so that a value of 3 is correctly identified as "None" instead of "Inactive".
• Calculation of staff wins/losses/ties/playoffs/bowls corrected to catch some missing data. Only coaches who had spent time as coordinators were getting record data set correctly previously.

4.1.3 (March 18, 2006):

• Interrogator now has 2 new columns in the Staff.csv file:
  • Role: Current role on current team. If this is "3", the staff member is no longer active in the game.
  • Retiring: If this is "1", the staff member has retired this offseason and while visible in the game is not available to be hired.
• Interrogator has split off a new file from DraftOrder: DraftStatus.csv. This has information about any ongoing draft. DraftOrder.csv should now have the correct team's assigned for the next draft (or current if ongoing) and info about future drafts (traded picks are marked here).

4.1.2 (February 26, 2006):

• GameViewer has another modified layout. Comments welcome.
• As part of the new layout, added a new menu item "Options" to GameViewer. Right now it only has one entry: "Formations on Field". If checked this will cause the play text to skip offensive / defensive formations, and these will appear on the field instead.
• Fixed a GameViewer issue with no team for a receiver who fumbles the ball.
• GameViewer now colors the stat lines for the away team with a light gray background to differentiate them from the home team stats.

4.1.1 (February 24, 2006):

• LogFileProcessor now indicates when it is finished processing.
• GameViewer now handles a few more penalties:
  • Illegal Chop Block
  • Ineligible Receiver Downfield
• Fixed determination of which team the penalty was on in GameViewer (which led to some improperly negated plays for stat collection).
• Fixed some GameViewer issues with determining punts inside the 20.
• GameViewer now treats kneeldowns as rushing attempts.
• GameViewer now handles stats for the final play of the game.
• GameViewer has a modified layout. Comments welcome.

4.1.0 (February 19, 2006):

• Interrogator correctly handles custom city lists.
• Game Viewer now starts each game paused.
• Game Viewer now tracks a bunch of stats.
• Game Viewer now animates the ball during each play.
• LogFileProcessor now shows progress as it parses each selected log.
• LogFileProcessor now has a "Summary" option that dumps a summary.csv file with some collected information about each game. This is used for occasional statistical checks, such as what impact weather has on each game (which prompted this feature).

4.0.2 (January 16, 2006):

• Interrogator Fixes:
  • Fixed calculation of player indices in the transaction log.
  • Guarded against out-of-range errors on staff attributes.
  • Match correct players up to historical records.
  • Remove "Year Retired" field in the PlayerRec2 fields as unreliable to calculate.

4.0.1 (January 15, 2006):

• Game Viewer now shows the status of the current drive (plays, yards, time).
• Game Viewer now supports endzone and field logos for the home team. See the documentation for details.

4.0.0 (January 14, 2006):

• New Utility: FOFGameViewer. This is an early release of a new graphical game viewer. It is usable, but is missing stat tracking and planned on-field animations (formations, ball movement, etc.
• Interrogator now deals better with retired players. Figures out what year a player retired (written to the PlayerRec2 file) and accounts for those players when reading stats for a year.
• Interrogator now dumps out Hall of Fame fields into the PlayerRec2 file.

3.2.0 (December 20, 2005):

• Now built with Visual Studio .NET 2005 final, so no more beta runtime!
• Fixed calculation of player indices for the most recent year's PlayerGame file in Interrogator.

3.1.1 (October 25, 2005):

• Interrogator now gets injury names directly from injuries.fdt (d'oh!)
• TeamData.csv now includes team nickname and city index

3.1.0 (October 22, 2005):

• Interrogator now writes out:
  • Agents.csv with a list of all player agents
  • LookupTables.csv with some int-to-string mappings for various fields
• Cleaned up and split up various other fields to reduce/eliminate decoding

3.0.3 (October 19, 2005):

• Interrogator now writes out:
  • GeneralManager.csv with year-by-year General Manager scores (most useful for single-player careers)
  • Franchise.csv with year-by-year franchise data
  • GameResult.csv with the results of every game ever played in the career
  • Schedule.csv with this season's schedule of games.
  • Cities.csv with current data on all cities capable of supporting an NFL franchise
  • PlayerGame*.csv (one for each season) with game-by-game stats for every player that played that season 

3.0.2 (September 16, 2005):

• Interrogator now writes out Scouts.csv with all relevant information about scouts in the game.
• DraftAnalyzer now has a minimize button. That's only about 15 months late...

3.0.1 (September 7, 2005):

• Interrogator now writes out Staff.csv and StaffHistory.csv. Staff.csv contains all relevant information about coaches in the game. StaffHistory is a year-by-year breakdown for each staff member regarding what team they were on, what role they held, and how that team did that season.

3.0.0 (September 3, 2005):

• New utility: FOFInterrogator. This will parse an FOF2k4 .FGA file and write out several comma-delimited files with user-visible data about that league. This is a work-in-progress, but writes just enough tables to be useful now. More tables will be added later.
• New utility: FOFLogFileProcessor. This will give you a list of game log files in a specified directory, and will convert any you select to CSV files that have a row for each play. Should be useful for finding out what play calls are working and which aren't.

2.4.2 (August 10, 2005):

• Fixed an Extractor bug where the columns varied for each team depending on the players on each team's roster when extracting all teams. For example, if there was an unsigned rookie on a team roster but not on the free agent list (which comes first), the column headings and all teams until then will not have draft data, but all players on that team will. Now, when extracting rosters for all teams, all columns will appear in the output regardless of what players are in the list (so entire sets of columns can be blank). This is only when dumping all teams.

2.4.1 (June 14, 2005):

• Extractor can now extract all teams. Open a team roster window, then hit the extract button. Extractor will automatically step through the teams, open the first player in the list, run through that team, close the player, and move on to the next team. You do not need to open a Player Report window (and if you do, you will only get that team).
• Extractor now pulls the birthdate and team for each player. Check your spreadsheets / utilities.
• Analyzer updated to work with the new Extractor files as well as the old. However, it cannot load old Analyzer saves, so wait to upgrade if you are in the middle of a draft.
• Analyzer will now display a player's birthdate, if it is available from the Extractor file.

2.4.0 (May 22, 2005):

• HTML Export of the draft class now colors in the combine numbers in the same way the Draft Analyzer UI does.
• Extractor now puts a header line at the top of the output files to indicate what each column is. The only exceptions are for the position attributes at the end of a player report, which are position-dependent.
• Extractor will now grab salary data if available.
• Extractor will now note what position a player is a mentor to, if any.
• Analyzer now has a button and drop-down that allows you to sort the players by a specific skill (Run Defense, Avoid Fumbles, etc).

2.3.1 (May 8, 2005):

• Fixed a bug in the way the College Names file from TCY is opened if using TCY HTML imports.

2.3.0 (May 7, 2005):

• Analyzer has a player details window that shows more information on the selected player. This includes height, weight, college, detailed attributes, and valid positions for the player to switch to based on their weight and college position.
• Analyzer can now read in HTML from the TCY HTML export when utility when loading a draft class from Extractor. When importing a class, it will pop up a dialog that will allow you to specify paths to the various HTML export files. If you hit Cancel, it will skip the import. Otherwise, tell it where all your HTML files are, and you'll get stats from TCY in the Analyzer player details window.
• Instead of calculating averages and standard deviations for combine scores from the current draft class, Analyzer now uses standard values gleaned from looking at a bunch of draft classes and generating averages and deviations across the entire group. These values will be more accurate now. This is subject to tweaking in the future if I manage to collect even more data. Thanks to Tium for the original work to generate this data (see the following thread at FOFC: http://dynamic2.gamespy.com/~fof/forums/showthread.php?t=21765&page=2&pp=50).
• Analyzer now uses coloring on combine scores to highlight players with particularly good or bad scores. Light green is used for combine scores at least 1.5 standard deviations better than the average for that position. Dark green is at least 1 standard deviation better, dark blue is at least 1 standard deviation worse, and light blue is at least 1.5 standard deviations worse.

2.2.1 (July 3, 2004):

• Extractor now correctly reads "Bowl Wins" for coaches.
• Draft Analyzer now separates position into its own column in the mock draft.
• Draft Analyzer now lists which pick number in the mock draft each player is in addition to his place within the round (i.e. the guy picked in round 2 pick 2 shows as 2 (34) in the pick column). Lost picks are skipped for this.
• Draft Analyzer now has an "HTML Export" button that will save the main list as an HTML file, with drafted players marked in red and all scout-based columns pulled.This is another way for multiplayer league commissioners to keep GMs up-to-date with how the draft is progressing, especially those not also using Draft Analyzer (and who therefore can't use the draftee lists). Idea for this shamelessly stolen from the inaugural WigFL draft, and therefore this feature is hereby nicknamed "WigOut".
• Cleaned up various HTML outputs from DraftAnalyzer to remove redundant tags among other tidbits.

2.2.0 (June 27, 2004):

• Draft Analyzer now has a "Mock Draft" feature. Check out the Analyzer documentation on how to use this feature.

2.1.2 (June 21, 2004):

• Added more weights to the Draft Analyzer Weights dialog to give you more control over how the draftees are weighted. This is primarily intended to allow you to adjust how important the future rating, percent developed, and combine scores are as the draft progresses.

2.1.1 (June 19, 2004):

• Draft Analyzer now selects all text in the copy/paste area whenever the text box is clicked on. This should make it easier to paste over whatever is in the box already.
• Draft Analyzer can now handle more than one line of text when pasting an FOF draft. Select a whole bunch of draftees in the FOF2k4 window, copy, then paste into Analyzer and you'll get them all marked.

2.1.0 (June 16, 2004):

• Extractor now exports a player's college in roster lists.
• Draft Analyzer allows setting and editing of a draft order. This field is exported to the draft list when saved.
• Draft Analyzer can now read back a saved draft list for further processing.
• Removed "Debug First Player" checkbox. The implementation was dependent on GDI+ which many users don't have.
• Draft Analyzer is set up to track FOF drafts. See docs for full details.

2.0.1 (June 3, 2004):

• Draft Analyzer screen now adjusts better to different screen font sizes.
• Draft Analyzer now clears its lists when loading new files, so you can load new draft lists without restarting the utility.
• Draft Analyzer now displays / outputs player names in "Last name, First name" format like the game uses in its lists and draft.
• Draft Analyzer now sorts on any column. A stable sort is used, which means that if two rows have the same sort value, they will remain in the same pre-sorted order relative to one another. So you can sort by strength, and then by position, and within the position groups everyone will remain sorted by strength.

2.0.0 (May 30, 2004):

• Can now export scouts and coaches. The program can detect what type of report window is open and act accordingly.
• HTML documentation and installer provided.
• New utility: Draft Analyzer. Has some new ways to rank draft players, and allows customization of the weights it uses.

1.2.2 (April 14, 2004):

• Fixed extraction of "Current Impression" from draft screen. Forgot that "I" and "l" look the same to the font.

1.2.1 (March 28, 2004):

• Added "Debug First Player" checkbox. When checked, the program will dump diagnostic information for the first player encountered. This includes 3 bitmaps (PlayerInfo1.bmp, PlayerInfo2.bmp, and ScoutingReport.bmp) that are the screen captures the utility uses to process the player, and a "FontData.txt" file that has the calculated information about the font used on the system for running the OCR algorithm. The BMP files may be useful to a user to see if anything is blocking the player data, or all four of these files can be zipped up and e-mailed to me in lieu of a screenshot to give me more debugging information. If a particular player is giving problems, be sure to open that player window first before running the Extractor.
• The dialog box for the app now opens in the upper-left corner, reducing the likelihood that it will be blocking the game window. I'll probably get around to adding some memory to this routine eventually so that it will always re-open in the last place you opened it.

1.2.0:

• Re-wrote the OCR initialization algorithm to render the characters on the user's system using their font setup. This means the utility can automatically differentiate between large and small fonts, and more easily handles a larger character set.
• Removed the "Big Fonts" checkbox as no longer necessary.
• Added several new fields that were much easier to detect and deal with thanks to the new OCR initialization. See the readme for the new column layout.
• Handles mixed draft/roster data. This typically happens when dumping the free agent list, or if your team has unsigned draft picks, or if it's between the draft and training camp.
• Added a check to see if the FOFExtractor dialog is on top of the Player Roster window. This is the cause of about 2/3 of the error reports I get.

1.1.3:

• Fixed stupid bug introduced in 1.1.2 that prevented the tool from ever reading Volatility and Solecismic scores for draft lists. It will still catch the old lockup case and display an error instead.
• Improved the error handling code to allow the user to decide between aborting (CANCEL) or continuing (RETRY).

1.1.2:

• Added some diagnostic information to display what field is being worked on. This can help narrow down issues people are having running the tool.
• Displays an error dialog when it tries to extract numbers from fields that are displayed as a description followed by a number in parentheses. For example, Volatility might be shown as "Average (48)". In the past the tool would lock up if it thought it was parsing a field like this but wasn't seeing that data.
• Tool will now abort when it hits an error, rather than trying to continue processing.

1.1.1:

• Fixed a bug that interpreted floating-point values with a zero after the decimal place incorrectly. "5.03" was interpreted as "5.3".
• Support for large fonts (or the 120-dpi setting in Windows XP). The app will not auto-detect this, but does provide a checkbox for you to click if you need large font support.

1.1.0:

• Support for any player report window, including those from your team's roster. Note that the format of draft reports and active player reports differs in that the player reports do not have combine ratings and one or two other fields.
• Improved color detection for 16-bit desktops.

1.0.0:

• Initial version. Supports draft data only.

Obligatory Warning

I've done my best to make sure that this program is virus free and safe to install. However, I cannot make any warranties for how it will behave on any given system, nor do I accept responsibility for any problems that result from downloading or running this program. Use at your own risk.

Greg Stelmack, webmaster@gstelmack.com