A long awaited techological advancement in scoring tests and surveys with less than one error in 1.5 million marks!
Error rates as high 0.5% is unacceptable for students taking SAT and other tests. Error rates near 0.01% is the best ever achieved by standardized scoring systems using Optical Mark Recognition (OMR) technologies. AVANTE has pioneered the use of optical imaging technologies with innovative use of scaling and quantitative pixel measurements to achieve accuracy rates of 0 errors in 1,500,000 marks! This rate was certified by an independent testing laboratory for Federal election applications.
SOLUTIONS FOR FIXING THE COLLEGE BOARD SAT SCORING ERRORS
The recent scoring errors of SAT by the tabulation company engaged by the College Board highlighted the long needed technology upgrading. The errors were attributed to the moisture induced paper expansion and inability to resolve lightly marked “bubble marks”.
The inability to resolve lightly marked “bubble marks” and errors due to paper expansion or shrinkage are known problem for at least 30 years since automatic tabulation of standardized tests was used. When combine with the inherent paper alignment errors, as much as 1.5% of tests were tallied wrong. For the individual taking the test, the error rate is as high as 400 points or 25% in a 1600 points test.
Interestingly, the same method and error rates also appeared in tabulation of votes in US elections when paper ballots are used. In fact, the Federal Voting System Standards of 2002 specifically mandate less than 1 error in 500,000 marks when the paper ballot is marked correctly (i.e. properly filled “bubble marks”). There is no proven solution capable to pass the testing requirements based on the more traditional discrete sensor-OMR technology that is also used commonly in scoring standardized tests.
There is a solution recently developed for tallying ballots using pixel count and document imaging technology by AVANTE International Technology, Inc. located in Princeton, NJ. It is the first to pass this stringent federal testing with zero error in tabulation in 1,500,000 marks. The same technology has been used successfully for tabulation of surveys and questionnaires besides tallying ballots.
The following compares AVANTE's Test and Survey Scoring System and the conventional OMR and discrete sensor mark-sense systems.
|
COMPARATIVE ANALYSIS OF DISCRETE SENSOR/OMR AND PIXEL-IMAGING (AVANTE Patented US 6,892,944; 7,077,313 and other patents pending) |
||
|
|
DISCRETE SENSOR-OMR |
IMAGING + PIXEL COUNT |
|
Recognition of filled “bubble mark” |
Yes |
Yes |
|
Recognition of lightly filled “bubble mark” |
Mostly no (Need high threshold) |
Yes |
|
Independence on timing tracks |
No |
Yes |
|
Ability to resolve registration (fiducial) marks |
No |
Yes |
|
Ability to resolve wrinkled and creased papers |
No |
Yes |
|
Ability to resolve paper shrinkage/expansion |
No |
Yes |
|
Recognition of “þ” and “ý” marks |
Mostly No or marginal. |
Yes |
|
Resolving barcode (type or individual form) |
Yes |
Yes |
|
Resolving multiple and different pages |
Yes |
Yes |
|
Resolving random orientation of pages |
No |
Yes |
|
“Separating” written answers from “bubbles” |
No |
Yes |
|
Automatic self-check for accuracy |
No |
Yes |
|
Ability to retrieve individual test for recount |
No |
Yes |
|
Speed (Per Scanner) |
Up to 10,000 pages/hr-scanner |
Up to 6,000 pages/hr-scanner |
|
Accuracy |
> 1/1,000 (2006 SAT case: >1/100) |
<1/1,500,000 (Federal ITA tested) |
|
Comparative Analysis of the Traditional Discrete Sensor Based “OMR” Tabulation System And Pixel Based Optical Imaging Tabulation System |
||
|
|
Discrete Sensor Based “OMR” System |
Pixel Based Optical Imaging System (AVANTE patented innovation) |
|
1. Ease of Audit |
§ Answers are rated by a pre-set threshold of “YES” or “NO” as the forms are scanned. § No automatic audit can be done without complete re-scanning with another setting. § No electronic images of the forms are kept. Original paper can be used as a last resort. |
§ Forms are first imaged and stored. § Actual pixels in each of the marked answers are measured. § “YES” or “NO” is set at 20% default. Higher or lower settings are used for automatic audit for any possible errors. § Electronic image may be pulled for any specific form for manual audit. |
|
2. Costs of forms and readers |
§ Precision printed forms cost more. § Specialized form readers are more expensive. |
§ Standard office papers are fine. § Office laser printers with more than 200 dpi print quality is adequate. |
|
3. Multiple page forms |
§ Barcode identifiers for pages are used and able to handle properly sorted forms. |
§ Barcode identifiers for pages and forms are used and able to handle forms without pre-sorting. |
|
4. Stacking and alignment |
§ Forms are preferably pre-stacked and aligned. § Some ability to handle mixed forms. |
§ Fiducial marks and barcode identifiers eliminate need for pre-sorting. § Even wrinkled forms are OK. |
|
5. Flexibility of forms |
§ Marks must line up in columnar manner to match the sensor columns. § Absolute marking positions against the edge of the paper must be maintained or errors result. § Precision printed forms must be used. |
§ A set of fiducial marks is printed for relative position of marked space. § Standard laser printed paper forms are acceptable. |
|
6. Accuracy |
§ Standard election applications show 1 in 1,000-10,000 errors. § More carefully marked forms get better results. § High moisture or extreme dryness may expand or shrink papers to cause substantially more than 1% error rate. |
§ Demonstrated less than 1 error in 1,500,000 marks. § Only 20% volume need to be filled to correctly register. § Patented incorporation of fiducial markers and automatic scaling to correct any paper shrinkage or expansion including folding marks, creases and wrinkled papers. |
The discrete Optical Mark Recognition (OMR) sensor technology relies on completed documents passing through the scanner “exactly” straight. The width marking positions must line up “correctly” and timing must be exact for reading the length of the paper. With the use of advanced document imaging technology, AVANTE uses fiducial markings to scale for paper orientation and expansion/shrinkage variations. The quantitative use of counting pixels in each “bubble mark” also provides automatic self-checking for possible light markings or other smearings or errors. The following table is a summary of differences between traditional discrete sensor OMRs and imaging with quantitative pixel counting technologies.
Previous page: AVANTE Patented Scoring Technology
Next page: Test Generation & Scoring