ࡱ> CG,-FEDzakW7{y3T( n$9]+?Pe~.*)PNG  IHDRgAMAPLTEٟ IDATxU1P{$@|q@gxY;93+q'O~H@cǫ:q55u6RBWB}ު^'{P$>}|_o}i/#/#/#/#/#?"q4q4 NÌD\ȺBdoRnᨪ@d>kP(f5Pcf$dZ޽42 ZЂaH"ai/<t 1a$RX6PCiƚ2l``J$F@L$0{ *swy ,Ji@GJcՊ~f$)y=F!T$3"u/Z3VIH9b?5^>"M_$VJRE Y?%=:21H]`BWUgzEc74$y'<3ծ|t )upz7ɒF`EC?fĬR|z$7WPxZs;6!Bcn)`,B)d8eEYl %6o 0o:(q#$\|:$y8 ;E#$ц?Qvz)y9Sj< GD f%i=eƼ QT9eɌvBm(+_G#blY ) mK|G"U<`2h"* 6i<0,2,l`쑈pLlxd3hnEp&2"X Æ~1  OGCccd"ρr r =>"|JʙEQHVA'^M-b\ t2R.2e \&Z4 caU#RKH%FA*V fBX.3r>Iz~r,7r$a&Z ]b 2\jD]P*jFOUH C]1Q Jc&q^.r|=Fpa"Bx ԱjE̠r>agMs>"R~feBrPjzPPV\wbeaCiRu}!8vBc26M q`oD4CkA'~!b NDDl7YC+_+$ -y}KW1s[0a:clڀ B&4" .j ցTRenm09~ M=p(A fUwbʢ0 SNj 4P`2aNS ñʮ #U`<^<zUN8FlF!X<r+CBaB9ɍ͆٦CHRS_PQXF^bL&B" *6PPOwDGPV9UP_YEt{JRU59vHTFp< !V^LcE+I@Hm:@CMC|N$Է9nm N%' )D! =݂cmDbI͹$ "G#$@e@`!Ng3S݆Ŝ?Eo x, !kE*Ra`۴$'D ,?E *nC$>u̍\r`CX `[sIA " i1tbݎ7kvN?@#odO4D0Cx!$RψKP yH%F xZZ~8z]03`&8F.@VC,!L+`io3PGHAdaF׾=#$u[xf51uЕ`"4c ,@jXj # `$8a!;kU`rb2qm$1|,$Rd-# *^'m*SlD! rJ }ix0P2%Sg`sr>ȼɢ6<"!kA*Dp /z04 T"y|l IL!3"5`$|3,1L2%v"/4!rPе`B#ԌtOt@vOGߚDfd_X+B=z((wӧ~[hIX)"('^z>P셊- j PWDO&4x-d`z|C?gmqjvBSb==zB9zv#f\o!5L)s+$Bp֙`yRJ -i`* W)Y-`G|_ú567A"т"xOA];!1Cݞg )&A`-s{Hm|b%#E}{ |ԅ~1I!|̈5-F^ (n' /hy,hO.{>ˍ‹d!іe/U>"݅ <@Lr[l.՗;k#A,b<eZW# S{Bxa`5f`ag5Wը*Rd ļ %fX– b^B=I9Pp o-0g!&ܥ=;kaDmZH!G""Fv |NDN))FɃx1iT#O]t*%PZqh l)C"[Q6B1'aUZ⨹iשY5R걓ӌnH39}$Cd!;4Ө m %RX׾U!ך_VMH8M)B5j5N4o܉^J9襉 X9jqxS~8Cڼ*"h@ZR1FNcBW\,$MjEvѐcOcz'b%}$rП KeIvEGoAV5Q%Y^OL,o_{l^S/pnB:iBȡFӛH.vq7HMB&s9!rޥ1+"c Fm]AQrPxG6ϟM!?H"U\1ղkE&~sc>"k`t ƣvx?_0(Byku}ϖC4Z X'ݞj \4"o毦4mKMjӡ 1Kt%؋d`aP?M~q2cFRi죆ʬ^놽qGy5w>J Hp/><ݮԫ*DԼK{\~nz(*Q 2kryQο*wiB џ7/g.8P} r*סCt4 [o^EUZNK`{0R\VzSyչڥ_YqxI{s_\۵ZQ'$d4B:5LNc4uo׶_{TSD%8Pt ZqWðW{1IDX,t98WDMHt$E\du̿姩 _?F$2De'7zԗU? bR3d[iC6A';=/OG_X]+]~K)"Mm@ w F2 )# yĸNMpW>;l$OwA!̣:aLWʸc:l;ڻC鋖 ^}k}OP.|n%+k)B|_Ռ١_+o\>l$$E($?wRDirܯаgv{pg~"F)?bŞwu/Pʈ{ d1{2'NER,]ι//{w?3n ROe<~Ez+5fo=7:A!UL40To6w9)R`ra!:! 0q?E I/X݇io諟4uꍻ.9IO";9Bb>|ͼtoǣto)#ZɂEI"QM=cN?۹;66&V1Z(k$ /+6䡙ߘNif#F䲍 7_ذ F}U)gYQ*!ΧF3{])vQ<S0v߰vk_kߺ߷DڥJ(m>5eAKvOfH rQISt8l՚gr iQH(EҶ|䆷*^)2l[F̣cnvڃKlerhبNH }rÿ>&}=>3EH 0!ːgn;Uxm_oh# !+J<;l'>෡5ټ73ڲ``6q=汦ջF׾飙0<^1/ƈH6bӒt$N+pd(ڍ\r͑2-p6{|M7Xܓ;RG|""U-pT;8|-y膫H};ϐiˈs]s CcV>l9+(lSO1eȱ6/t!8qgH3 F6Fr׽{qex Te 56n̫~X>EҥcH0~tH5׾`Z@o!iݔ"r{^GJlxKMxQQ)fT]E$.2'!~Pːvq:C2% q1dh|ݟkWGlb.!۠=~\ꟃ]ϐ f{6i;-'K]p{CvO)HdsJƦ>n ʷi4my0{H9Rqve2z~ek_"mmty32DlL1tK(\CA3To0Z-zjR7&K T &hLSY%y5SsϾa]R`|e`$$"u~#.6lڛEdtzOw7h1UӶjbH+-A[;8mBӱ#sqܾ2֬cU{? hZb`F 58?]}K?}˳5Ռqc!B{6Dt oo_~#S *M[Wd ^o]ǎ]yS?WNZYxNڷh,RCN}[Y`bZMp6Gn*ҹԨ$WП js"߼=VOgɷ}^4y ~(QeH82,|_ /; jHoA>T_4.?徫V!-1~ㄚrdIrfgH$ 8WBaJAh%%eD0ZU+V hiT٫|ȽI3 r:4&oAyq0i;68THp!|3[BoxdJګ!-O33ߕ(ь!U}󃈣)K׳‡+3ᴡ(°*".ģ0尨!F?<;$`)#H3U.ـТBD[nĿJp(H}C^J83$yHa G#0?>2-SNdcEȍBx!E[r|̡^\C4&z쇙z$TGJKu?E|-1?xY`i^_{(Ҳ1#ZAXq ЕZH/eW@GN TB,N){JS$qea#qe#!" $ ^&W?LUIO+l*\<лc ,vzʲ-#{cT/7Np`̒KtJ<3!+04Vf8{p^SZJ'`❧HY[b6šAĵ)˫q̣.2'Q =-eA\5JnE% \ܔ/e`ALe{HKDAwo\@+k}G贐& ljf{OoeI pS}T2׋J4՜dY\5ݶ1UڦGZqn^KL"&i{3=BDBOR}bB^NЙ: B;4Ɛ[fs3IޟeH~zgE' {9 {uCB2s}$ y`^56jBX4icW.ٚ쟙#r 2401BcHjE}L+樺eDGdm4z  Ybt3U,S V!^s9k }m^!hŲ8\̃CcƀW2B .  =cUH13Qi6{iKDZ! 71mQ24}Ta| DVAlbdX pn)zk5ՐΘ gm#EKtw!A `0@Hيe23)r qN:)cԱ*ЩЖO bZ3Jw 2@b te05ELJ й@ڙoD"@(*pAF"%V !*c( 288&AȢ"mʐ]H @P_a|t)fO@A:\9BgT?ZRdGjP@QuZcz:! ?Ij!@aƜZ7 -@Ԟl+β iSxcj3nl\r,P: n00Hju } X!EkV8`i970ZW8!1(?G7XNi:9jFь`aP0@^f HC9X~$N? 3 !ra+aaD2:IF b><=/>XH,ZŴOV !GרWF UfwC _9:ȋ/P P\{+_C "~=M+BGC&m/5@>:x@P]*N^ЉKf1K"1$ l;^4}f42JXN'3 k }m:/ |BO|n@B)җ /\H3b(d$"|=yI69WӉN~#^F~vDu2NzFїIENDB`( #  thttp://www.nysedregents.org/testing/socstre/ushgrg.605.pdfthttp://www.nysedregents.org/testing/socstre/ushgrg.605.pdf#4emscreminfo@mail.nysed.govBmailto:emscreminfo@mail.nysed.gov^Dwww.p12.nysed.gov/assessment/hsgenRhttp://www.p12.nysed.gov/assessment/hsgen_Dwww.p12.nysed.gov/assessment/hsgenRhttp://www.p12.nysed.gov/assessment/hsgenlmhttp://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdfhttp://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdfnhttp://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdfDohttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfDphttp://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf xhttp://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfmxhttp://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfmvhttp://www.p12.nysed.gov/assessment/sam/secondary/home.htmlvhttp://www.p12.nysed.gov/assessment/sam/secondary/home.html,dmerlau@mail.nysed.gov:mailto:dmerlau@mail.nysed.gov,gwilsey@mail.nysed.gov:mailto:gwilsey@mail.nysed.govF/ 0|DArialgsRomanPP8xؖF0ؖDVerdanasRomanPP8xؖF0ؖ" DTimes New RomanPP8xؖF0ؖ0DWingdingsRomanPP8xؖF0ؖ@DTahomagsRomanPP8xؖF0ؖ" A .  @n?" dd@  @@``   T= 21   () 5  >>    :<=C       tuv|}             ?r$kW7{y3Ti$b$9]+?Pe~.*)$i 0AA@8gXhY) mS ʚ;l8ʚ;g4PdPdXϘF08ppp@ <4dddd@0Px<4BdBd@0PxX <4!d!d(A0P0___PPT10 f___PPT9H@}~u[/01)mn@KGLHI<=fg89:;^M *e Tdpqryz{|c?  %$bh The 鶹ýӳ State Education Department Demystifying the Holistic Scoring Process in Social Studies: Hi$($;lRegents Examinations in Global History and Geography and United States History and Government Spring 2013 "mZl{tHolistic Scoring OverviewWhen scoring holistically Read thoroughly, yet quickly to gain an impression of the entire response Read the entire response before determining a score, and then promptly assign a score J" " }Scores must be determined based on the holistic application of the content-specific rubric provided by 鶹ýӳ. The anchor and practice papers serve as examples of what a student response at a particular score level should look like. The content-specific rubric is not a checklist. " Z~ Important points to keep in mind!!"The quality of handwriting, the use of cursive or printing, margins, editing marks, cross-outs, and overall neatness are not part of the scoring criteria, unless the response is completely illegible and nothing can be read. " |u(It is important for each reader to set aside his/her biases in order to keep the scoring as consistent and as fair to each student as possible. You are scoring a State exam, not grading a classroom assignment, therefore, personal classroom grading practices should be set aside while scoring. )" Z)_[@The Six Primary Scoring Criteria for the Commencement-Level DBQAAThe extent to which students develop the task utilize higher level thinking skills use the information provided in the documents incorporate relevant outside information in the DBQ essay support ideas with facts, examples, and details organize and develop a social studies essay*" " 7No one bullet can hold a particular score point hostage88"QNumber of Documents Minor Errors Outside Information Copying Document InformationR" RDANumber of DocumentsnCan a paper still be given a score of 5 when the student did not use the minimum number of documents required?oooYes, because it is only one of the criteria on which you base a score. If fewer than the minimum number of documents are used and all other criteria of a 5 are met, then you may have a 5 paper. However, when fewer than the minimum number of documents are used, you often have other criteria that are not being met, and thus you usually end up with a 2, 3, or 4 paper.p" p5/ An example can be found on the August 2009 Global History and Geography DBQ. The Level 5B anchor paper uses 3 documents, instead of the minimum number of 4 documents. This can be found online at: http://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdf P" " mc0EC Minor Errors "60KCan a paper be given a score of 4 or 5 if it contains a minor error or two?LL"If the paper clearly demonstrates that the student understood the task and meets the criteria for a score of 4 or 5 on the test specific rubric, then, the paper can be given that score. A paper does not have to be perfect to reach a score of 5. " 71<An example can be found in the June 2010 United States History and Government DBQ. In this Level 4A anchor paper, the student states that with the passage of the 18th amendment alcohol consumption was made illegal. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdf 6" Z" Z" Z" ZZZ nc05FDOutside InformationHow does  outside information fit into the overall scoring of the essay?JJ"The 4th bullet must not be overemphasized in scoring. A response may receive a 3 without outside information. However, a response with outside information is not automatically a 3.*" -)The quantity and quality of the outside information, the relevance of the outside information to the topic, and how it has been integrated into the essay should be used when determining the score of a response. 6" ZZZsmOften outside information can improve the overall essay when it is used to make historical connections and show analytical thinking. ." ", All papers need to be evaluated based on the rubric. If the paper demonstrates that the student minimally develops the task, is primarily descriptive with weak or isolated information, has a general plan of organization, and includes outside information, the paper is probably more of a 2 than a 3. "," -tnAn example of a level 3 response that uses no outside information can be found on the June 2009 United States History and Government DBQ, Practice Paper B. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdf 2" " pc0G@An example of a level 2 response that uses outside information can be found on the January 2009 United States History and Government DBQ, Level 2A paper. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdf D" ZM" ZZoc0IFCopying Document Information How should you rate a DBQ essay when most of the response is copied directly from the documents and little or no original work is provided? " JK" A distinction needs to be made between paraphrased or copied information that is used to support a specific idea within the essay (an appropriate social studies skill) and the outright copying of information from the documents in an attempted response to the DBQ. 0 "  KGGenerally, when most of the information is copied directly from the documents without adding anything, the response will be rated between a 0 and a 2. 0"  LLMThe revised generic rubrics for the Global and USHG exams address this issue NN Score of 2:  or consists primarily of relevant information copied from the documents Score of 1:  or consists primarily of relevant and irrelevant information copied from the documents Score of 0:  includes only entire documents copied from the test booklet ," MHThe actual score will depend on three things: 1) the degree or extent to which the response has been copied, 2) the selection of what specific content the student chose to copy for inclusion, and 3) how the student integrated the copied material into the response. 0"  NIa Can a paper still be given a score of 5 or 4 when some information is copied from the documents?baxAn example of a level 4 response where the student selectively copied information from documents as well as included good historical outside information to interpret the documents can be found on the January 2009 United States History and Government DBQ, Level 4C paper. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdf 6," L" y@<(What criteria helps define a score of 5?))"KA 5 paper shows evaluation and/or analysis. It requires substantial outside information that is well integrated into the response. It contains historically accurate information. Document information is used to support the thesis of the response and is interwoven with outside information. It is cohesive and thoroughly developed.L" ZLA=A 5 is not a 5 just because it is the best response that exists out of all the responses being graded. A well-written response that contains numerous historical inaccuracies or repeats information over and over again is usually not a level 5 paper. The response needs to meet the criteria established by the rubric. On occasion, a particular essay question may have no papers at level 5." Zlf>What often distinguishes a level 4 paper from a level 3 paper???"cA 4 paper is both analytic and descriptive. It addresses all aspects of the task but may do so unevenly. It requires outside information that is integrated into the response. It contains historically accurate information. Document information is used to support the thesis of the response and is interwoven with outside information. It is well developed. (c" ZZdmgA 3 paper is more descriptive than analytic. It usually contains some outside information but it may not be well integrated. It contains supporting facts and details, but may also have some minor inaccuracies. "" <8>What often distinguishes a level 3 paper from a level 2 paper???"5A level 3 response contains more explanatory statements and specific details than a level 2 response. In a DBQ, the documents are used to support the thesis of the essay. A level 3 response should include outside information, but it may not. However, most solid level 3 papers do include outside information.6" Z6=9A level 2 response tends to have broad general statements. Use of supporting information and information from the documents is limited. Note: While a level 2 paper is not required to include outside information, it may contain some." >:=What most often distinguishes a 0 paper from a level 1 paper?>>"If a response is solely comprised of information copied from the documents and the historical context without any information provided by the student or accurately interpreted by the student, then the response is probably a 0. " B;A level 1 paper is NOT a paper in which the student simply  attempted an answer. It must have some appropriate, original information. *" rb^!Generic Scoring Rubrics by BulletContinuity of Score Points See Appendix E for Thematic Essays: http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf See Appendix F for DBQ Essays: http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf (" " c0?~c0QM Reminder:The rubrics define a set of scores on a continuum between 0 and 5. A standards-based assessment system measures where the student falls on that continuum. @" *%Scoring Related Questions'Issues Related to Scaffolding Questions((" How should you rate an open-ended question when the question asks for two reasons and the student writes two answers on the first line and a third answer on the second line, but one of the answers on the first line is incorrect? *  ?The general rule of thumb is that a rater cannot go searching for a correct answer. Disregarding incorrect information to get to a correct answer is not acceptable. The order in which correct and incorrect answers appear on the lines matters. The first two answers should be rated, even if they are on the same line. &?" @"&!$Issues Related to the DBQ Essay Task%%" How do you rate an essay where the task asks a student to write about two items in the historical context and the response includes discussion of three items? .*_In this situation, only the first two items that appear in the response can be evaluated. A rater cannot search the response to determine which of the three items the student wrote about were strongest. The rater must use the test specific rubric and must focus on information related to the first two items as they appear within the student response.`" Z`keHowever, there may be a situation where a student has appropriately applied information about the third item in a discussion on item one or item two. In this case, if the applied information supports the discussion, it should be considered when rating the overall response. $" +'Other Related Questions UWhy do some of the DBQ or Thematic essays use the phrase and/or within the question? :V9" " "  VSED has been using the phrase and/or on social studies essay questions since 1990. The use of and/or benefits students by broadening the scope of acceptable answers and providing a choice of content within the essay. And/or allows the rater to accept the broadest range of student answers as directed by the content-specific scoring rubric. W" Z""." ""t"  {" EXAMPLE: Discuss the political, social, and/or economic causes for the Russian Revolution. By wording the question this way, a teacher can accept a response that discusses any combination of causes. If the task said  and, the student would have to discuss one correct cause from each category. If the task said  or, the student would have to focus on one type of cause only and a rater could not give credit for information on a different type of cause. LZZ"""yHow do raters know how many examples a student should provide when responding to a particular DBQ or Thematic essay task?zzLook for the  S , the plural. This indicates that two or more examples must be provided. However, in the case of a DBQ, a student must also use a minimum number of documents in his/her response and support the essay with as many facts, examples, and details as he/she can. Thus, having more examples usually improves the essay. Collectively, the DBQ documents will always provide multiple examples. @" ZZ jd(If only one cause or impact is asked for, it will usually be phrased  a or  one . Example: a cause of& or one cause of& In general, teachers should teach their students to use as many examples as they can. It adds detail to the essay. It helps show the student s knowledge. " F fbOther Scoring Notes` In Global History, can we rate essay responses in which students wrote about the United States?aaMany topics in Global and U.S. History are intertwined. (The Cold War, The Middle East, Vietnam War, etc.) Not all essays will automatically exclude the use of information on the U.S. as part of an acceptable answer. An example of this can be found on the January 2011 Global History and Geography thematic essay. This can be found online at: http://www.nysedregents.org/GlobalHistoryGeography/20110128-gh-rg1.pdf *" ZZwpTools required for scoringDirections for Administering Regents Examinations www.p12.nysed.gov/assessment/hsgen H & G Information Booklet for Scoring Regents Examinations in Global History and Geography and United States History and Government www.p12.nysed.gov/assessment/hsgen School Administrator s Manual, Secondary Level Examinations http://www.p12.nysed.gov/assessment/sam/secondary/home.html .8" P>" Pv^c02T_c0c09txqGlobal History and Geography and United States History and Government rating guides for specific test administrations available online (" " yrTest manuals should be reviewed by all administrators, teachers, and staff administering and/or scoring a 鶹ýӳ State examination. Anyone who is involved in the scoring of a 鶹ýӳ State examination must complete the training as specified in the test manuals and rating guides." ZReminder2Teachers are allowed to proctor their own students, unless the district has a different policy. Teachers may not score their own students answer papers." y Training All training should be based on the protocols established by 鶹ýӳ as found in the H & G Information Booklet for Scoring Regents Examinations in Global History and Geography and United States History and Government as well as in the test specific rating guides." zScorers are encouraged to ask questions that relate to understanding the scoring guidelines and questionable papers, or scaffolding responses, but philosophical differences, personal preferences and biases must be set aside during the scoring process." {The goal of scoring is to be sure the scoring criteria in the rubrics are accurately and consistently applied to each student response being rated across 鶹ýӳ State." |-Questions about scoring social studies exams.."Contact the Associates in Social Studies in the Office of State Assessment For Global History and Geography: Donna Merlau, dmerlau@mail.nysed.gov For United States History and Government: Greg Wilsey, gwilsey@mail.nysed.gov Phone: 518-474-0556 rn" '" *" %" " " bu,c0}c0icYour input is importantFeedback on State examinations is very important to the test development process. This can be done by completing our online Teacher Evaluation Forms at: http://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfm B" @" " Ac0  0` !3̙` Q.<ffff3` 3333fff` 3K=̙fff` 3fffff` ff3ff3` aNR>ff` 3fY33` 3f3f>?" dd@&?oAd(@n<)o<6=nA+7%Z', n?" dd@   @@``PR   = 7 ,`(p>> ph (    < #" `j  T Click to edit Master title style! !P  0 "Pe<$0  RClick to edit Master text styles Second level Third level Fourth level Fifth level!     S^B  #  BCE6FGIIQU*VWX @`@$ G"  00`B  s *Dg "00  0 "^`  B*   0 "^   D*   0 "^   D*   dA"޽h ?Light horizontal"` 3f3f w ___PPT10W . "+FD ' ,= @B D ' = @BA?%,( < +O%,( < +D ' =%(D ' =%(D' =ABBB@B0B%())))?D' =1:Bvisible*o3>+#.<*!%(D' =ABBB@B0B%())))?D' =1:Bvisible*o3>+#.<*!.%(D' =ABBB@B0B%())))?D' =1:Bvisible*o3>+#.<*.:%(D' =ABBB@B0B%())))?D' =1:Bvisible*o3>+#.<*:G%(D' =ABBB@B0B%())))?D' =1:Bvisible*o3>+#.<*GS%(+8+0+0 + Profile4  X0 (    <\ #" `p  T Click to edit Master title style! !   08 "p` <$0  W#Click to edit Master subtitle style$ $  0p  "``  B*   0x "`   D*   0T "`   D* ^B  # BCE6FGjIQU*VWX @`@$ G") 00  dA"޽h ?Light horizontal"` 3f3f___PPT10. "+[W_D' ,= @B DJ' = @BA?%,( < +O%,( < +D' =%(D)' =%(D' =ABBB@B0B%())))?D' =1:Bvisible*o3>+#.<*$%(+8+0+0 + 8(    0, P   , >*   0     , @*   6, _P  , >*   6, _  , @* H  0޽h ? 3380___PPT10.y9@S<Y0N0  =(  ~  s *P  ~  s *P     0,@  '  (   0<P   ?University of the State of 鶹ýӳ State Education Department <?  2$j  C (ASEDseal3" PsH  0޽h ? f3y___PPT10Y+D=' ,= @B +} X0 $(  r  S j   r  S Pe  H  0޽h ? 3f3f___PPT10i.W+D=' ,= @B +} X0 L$(  Lr L S 0j   r L S Pe  H L 0޽h ? 3f3f___PPT10i.e Y0$+D=' ,= @B +} X0 P$(  Pr P S 'j   r P S X(Pe  H P 0޽h ? 3f3f___PPT10i.el+D=' ,= @B +} X0  $(  r  S j   r  S Pe  H  0޽h ? 3f3f___PPT10i.W0kҿ+D=' ,= @B + X0  P0(  Px" P c $|/j   x P c $ e  H P 0޽h ? 3fffff3̙3f̙___PPT10i.`{S+D=' ,= @B + X0  0(  x"  c $(|j   x  c $}PP  H  0޽h ? 3fffff3̙3f̙___PPT10i.]9B;+D=' ,= @B + X0 0 0(  x"  c $nj   x  c $wPe  H  0޽h ? 3fffff3̙3f̙___PPT10i.9a%+D=' ,= @B + X0N0 h(  ~  s *P`     s *P  <$0  H  0޽h ? f3___PPT10.+[W_DO' ,= @B D ' = @BA?%,( < +O%,( < +DA' =%(D' =%(D' =A@BBBB0B%(D' =1:Bvisible*o3>+#.<*p%(+8+0+0 + X0  0(  x"  c $j   x  c $Pe  H  0޽h ? 3fffff3̙3f̙___PPT10i.<+D=' ,= @B + X0 P 0(  x"  c $<j   x  c $Pe  H  0޽h ? 3fffff3̙3f̙___PPT10i.9[I+D=' ,= @B + X0  0(  x"  c $ԩj   x  c $Pe  H  0޽h ? 3fffff3̙3f̙___PPT10i.<)+D=' ,= @B + X0  0(  x"  c $Dj   x  c $   H  0޽h ? 3fffff3̙3f̙___PPT10i.<7+D=' ,= @B + X0 p 0(  x"  c $Dj   x  c $Pe  H  0޽h ? 3fffff3̙3f̙___PPT10i.9@B+D=' ,= @B +  X0N0 h(  ~  s *     s *Pe<$0  H  0޽h ? f3ia___PPT10A.+[W_D' ,= @B D' = @BA?%,( < +O%,( < +DA' =%(D' =%(D' =A@BBBB0B%(D' =1:Bvisible*o3>+#.<*7%(DA' =%(D' =%(D' =A@BBBB0B%(D' =1:Bvisible*o3>+#.<*7o%(DA' =%(D' =%(D' =A@BBBB0B%(D' =1:Bvisible*o3>+#.<*o%(+8+0+0 + X0 P T0(  Tx" T c $`j   x T c $ Pe  H T 0޽h ? 3fffff3̙3f̙___PPT10i.9 (+D=' ,= @B +} X0  $(  r  S j   r  S Pe  H  0޽h ? 3f3f___PPT10i.]`1~+D=' ,= @B + X0 *(  r"  S `@   x  c $PPP  H  0޽h ? 3fffff3̙3f̙___PPT10i.\9?+D=' ,= @B +} X0 0$(  r  S j   r  S dPe  H  0޽h ? 3f3f___PPT10i.]0+D=' ,= @B + X0  0(  x"  c $$j   x  c $ p  H  0޽h ? 3fffff3̙3f̙___PPT10i.8ʰ+D=' ,= @B +K X0  \(  x"  c $j     c $lPe<$0  H  0޽h ? 3fffff3̙3f̙'___PPT10..:v+[W_D' ,= @B DJ' = @BA?%,( < +O%,( < +D' =%(D)' =%(D' =ABBB@B0B%())))?D' =1:Bvisible*o3>+#.<*%(+8+0+0 + X0  *(  r"  S `p    x  c $Pe  H  0޽h ? 3fffff3̙3f̙___PPT10i.:+D=' ,= @B + X0  0(  x"  c $@ j   x  c $!Pe  H  0޽h ? 3fffff3̙3f̙___PPT10i.:gx+D=' ,= @B +} X0  $(  r"  S &`   r  S l'P`  H  0޽h ? 3fffff3̙3f̙___PPT10i.;  +D=' ,= @B + X0  0(  x"  c $1j   x  c $2Pe  H  0޽h ? 3fffff3̙3f̙___PPT10i.:0ux+D=' ,= @B + X0  *(  r"  S @=`   x  c $P6Pe  H  0޽h ? 3fffff3̙3f̙___PPT10i.:Py+D=' ,= @B + X0  0(  x"  c $Cj   x  c $H   H  0޽h ? 3fffff3̙3f̙___PPT10i.+D+D=' ,= @B + X0  0(  x"  c $^j   x  c $_Pe  H  0޽h ? 3fffff3̙3f̙___PPT10i.,̛!+D=' ,= @B +} X0 p$(  r  S \mj   r  S 4nPe  H  0޽h ? 3f3f___PPT10i.> hx+D=' ,= @B +} X0 $(  r  S X{j   r  S LPe  H  0޽h ? 3f3f___PPT10i.>pl+D=' ,= @B + X0  0(  x"  c $j   x  c $Pe  H  0޽h ? 3fffff3̙3f̙___PPT10i.)p.H+D=' ,= @B + X0  0(  x"  c $Dj   x  c $Pe  H  0޽h ? 3fffff3̙3f̙___PPT10i.*\[+D=' ,= @B + X0  0(  x"  c $,j   x  c $Pe  H  0޽h ? 3fffff3̙3f̙___PPT10i.*+D=' ,= @B + X0  0(  x"  c $j   x  c $hPe  H  0޽h ? 3fffff3̙3f̙___PPT10i.++D=' ,= @B + X0 P `0(  `x" ` c $   x ` c $tPe  H ` 0޽h ? 3fffff3̙3f̙___PPT10i. X+D=' ,= @B + X0  0(  x"  c $j   x  c $ȼPe  H  0޽h ? 3fffff3̙3f̙___PPT10i.Ł+D=' ,= @B + X0  D0(  Dx" D c $j   x D c $Pe  H D 0޽h ? 3fffff3̙3f̙___PPT10i.9+D=' ,= @B + X0N0 0 <(   ~   s *`0.   ~   s *8Pe  H   0޽h ? f3y___PPT10Y+D=' ,= @B + X0N0 @h(  ~  s *j     s *Pe<$ 0  H  0޽h ? f3y___PPT10Y+D=' ,= @B + X0 ,0(  ,x" , c $X0.   x , c $0Pe  H , 0޽h ? 3fffff3̙3f̙___PPT10i.~9M+D=' ,= @B + X0 ` X0(  Xx" X c $<j   x X c $Pe  H X 0޽h ? 3fffff3̙3f̙___PPT10i.9 +D=' ,= @B +} X0 P$(  r  S j   r  S Pe  H  0޽h ? 3f3f___PPT10i./r+D=' ,= @B + X0 0 L0(  Lx" L c $ j   x L c $ Pe   H L 0޽h ? 3fffff3̙3f̙___PPT10i.9`F +D=' ,= @B +  X0N0 Ph(  ~  s *      s * <$0   H  0޽h ? f3ia___PPT10A.+[W_D' ,= @B D' = @BA?%,( < +O%,( < +DA' =%(D' =%(D' =A@BBBB0B%(D' =1:Bvisible*o3>+#.<*S%(DA' =%(D' =%(D' =A@BBBB0B%(D' =1:Bvisible*o3>+#.<*S%(DA' =%(D' =%(D' =A@BBBB0B%(D' =1:Bvisible*o3>+#.<*W%(+8+0+0 + X0N0 `h(  ~  s * j     s *ZP<$0   H  0޽h ? f3y___PPT10Y+D=' ,= @B + X0N0 h(  ~  s *. `     s */ P<$0   H  0޽h ? f3___PPT10x.+[W_D' ,= @B D' = @BA?%,( < +O%,( < +D' =%(D)' =%(D' =A@BBBB0B%())))?D' =1:Bvisible*o3>+#.<*H%(D' =%(D)' =%(D' =A@BBBB0B%())))?D' =1:Bvisible*o3>+#.<*H%(+8+0+0 +} X0 @$(  r  S @ j   r  S A Pe   H  0޽h ? 3f3f___PPT10i. 0+D=' ,= @B + X0  x0(  xx" x c $P j   x x c $|Q Pe   H x 0޽h ? 3fffff3̙3f̙___PPT10i.+D=' ,= @B + X0N0 <(  ~  s * `   ~  s * Pe   H  0޽h ? f3y___PPT10Y+D=' ,= @B +} X0 $(  r  S  j   r  S  Pe   H  0޽h ? 3f3f___PPT10i.&+D=' ,= @B +$ X0 $(  r  S ж j   r  S x Pe   H  0޽h ? 3f3f80___PPT10.w$ X0 $(  r  S < j   r  S p Pe   H  0޽h ? 3f3f80___PPT10.$ X0 T$(  Tr T S H j   r T S  Pe   H T 0޽h ? 3f3f80___PPT10.eP_$ X0 4$(  4r 4 S  j   r 4 S  Pe   H 4 0޽h ? 3f3f80___PPT10.WPC/$ X0 8$(  8r 8 S  j   r 8 S | Pe   H 8 0޽h ? 3f3f80___PPT10.W`r$ X0 <$(  <r < S  j   r < S  Pe   H < 0޽h ? 3f3f80___PPT10.Wв$ X0 H$(  Hr H S 4 j   r H S  Pe   H H 0޽h ? 3f3f80___PPT10.W"} X0  $(  r  S  0   r  S  Pe   H  0޽h ? 3f3f___PPT10i.,8+D=' ,= @B +rT@.:0+6 r9xGfy#&!* W)- PA$50u<0C  e@0D@ SvI`ZvQ_bfDҜ[bB&9s w0JLN{ ԰cp&SRU~WYY޵P|# [^1( Oh+'0 hp  Scoring ClarificationsdmerlauProfiletnguyen282Microsoft Office PowerPoint@ O@0K9@gV Gh;  ,V'&" WMFCg <8What most often distinguishes a 0 paper from a level 1 paper? Slide 34"Generic Scoring Rubrics by Bullet Reminder:Scoring Related Questions(Issues Related to Scaffolding Questions Slide 39%Issues Related to the DBQ Essay Task Slide 41 Slide 42Other Related QuestionsVWhy do some of the DBQ or Thematic essays use the phrase and/or within the question? Slide 45zHow do raters know how many examples a student should provide when responding to a particular DBQ or Thematic essay task? Slide 47Other Scoring Notesa In Global History, can we rate essay responses in which students wrote about the United States?Tools required for scoring Slide 51 Slide 52 Reminder Training Slide 55 Slide 56.Questions about scoring social studies examsYour input is important  Fonts UsedDesign Template Slide Titles:  8@ _PID_HLINKSA `;http://www.nysedregents.org/testing/socstre/ushgrg.605.pdf"mailto:emscreminfo@mail.nysed.gov*http://www.p12.nysed.gov/assessment/hsgen*http://www.p12.nysed.gov/assessment/hsgenUhttp://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdfAhttp://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdfKhttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfKhttp://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdf@http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf@http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf=http://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfm<http://www.p12.nysed.gov/assessment/sam/secondary/home.htmlmailto:dmerlau@mail.nysed.govmailto:gwilsey@mail.nysed.govKhttp://www.nysedregents.org/USHist V thttp://www.nysedregents.org/testing/socstre/ushgrg.605.pdfthttp://www.nysedregents.org/testing/socstre/ushgrg.605.pdf#4emscreminfo@mail.nysed.govBmailto:emscreminfo@mail.nysed.gov^Dwww.p12.nysed.gov/assessment/hsgenRhttp://www.p12.nysed.gov/assessment/hsgen_Dwww.p12.nysed.gov/assessment/hsgenRhttp://www.p12.nysed.gov/assessment/hsgenlmhttp://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdfhttpRoot EntrydO)o)@PicturesY%Current User5SummaryInformation(      !"#%*0+./t23456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs$uvwxyz{|}_6tnguyentnguyen://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdfnhttp://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdfDohttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfDphttp://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdfhttp://www.nysedregents.org/  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~1USHistoryGov/Archive/20090616scoringkeyIII.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf xhttp://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfmxhttp://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfmvhttp://www.p12.nysed.gov/aPowerPoint Document(ZDocumentSummaryInformation8,#  thttp://www.nysedregents.org/testing/socstre/ushgrg.605.pdfthttp://www.nysedregents.org/testing/socstre/ushgrg.605.pdf#4emscreminfo@mail.nysed.govBmailto:emscreminfo@mail.nysed.gov^Dwww.p12.nysed.gov/assessment/hsgenRhttp://www.p12.nysed.gov/assessment/hsgen_Dwww.p12.nysed.gov/assessment/hsgenRhttp://www.p12.nysed.gov/assessment/hsgenlmhttp://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdfhttp://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdfnhttp://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdfDohttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfDphttp://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf xhttp://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfmxhttp://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfmvhttp://www.p12.nysed.gov/assessment/sam/secondary/home.htmlvhttp://www.p12.nysed.gov/assessment/sam/secondary/home.html,dmerlau@mail.nysed.gov:mailto:dmerlau@mail.nysed.gov,gwilsey@mail.nysed.gov:mailto:gwilsey@mail.nysed.govF/ 0|DArialgsRomanPP8xؖF0ؖDVerdanasRomanPP8xؖF0ؖ" DTimes New RomanPP8xؖF0ؖ0DWingdingsRomanPP8xؖF0ؖ@DTahomagsRomanPP8xؖF0ؖ" A .  @n?" dd@  @@``   X> 21   () 5  >>    :<=C       tuv|}             ?r$kW7{y3Ti$b$9]+?Pe~.*)$i 0AA@8gXhY) mS ʚ;l8ʚ;g4PdPdXϘF08ppp@ <4dddd@0Px<4BdBd@0Px <4!d!d(A0P0___PPT10 f___PPT9H@}~u[/01)mn@KGLHI<=fg89:;^M *e Tdpqryz{|c?  %bh The 鶹ýӳ State Education Department Demystifying the Holistic Scoring Process in Social Studies: Hi$($;lRegents Examinations in Global History and Geography and United States History and Government Spring 2013 "mZl{tHolistic Scoring OverviewWhen scoring holistically Read thoroughly, yet quickly to gain an impression of the entire response Read the entire response before determining a score, and then promptly assign a score J" " }Scores must be determined based on the holistic application of the content-specific rubric provided by 鶹ýӳ. The anchor and practice papers serve as examples of what a student response at a particular score level should look like. The content-specific rubric is not a checklist. " Z~ Important points to keep in mind!!"The quality of handwriting, the use of cursive or printing, margins, editing marks, cross-outs, and overall neatness are not part of the scoring criteria, unless the response is completely illegible and nothing can be read. " |u(It is important for each reader to set aside his/her biases in order to keep the scoring as consistent and as fair to each student as possible. You are scoring a State exam, not grading a classroom assignment, therefore, personal classroom grading practices should be set aside while scoring. )" Z)_[@The Six Primary Scoring Criteria for the Commencement-Level DBQAAThe extent to which students develop the task utilize higher level thinking skills use the information provided in the documents incorporate relevant outside information in the DBQ essay support ideas with facts, examples, and details organize and develop a social studies essay*" " 7No one bullet can hold a particular score point hostage88"QNumber of Documents Minor Errors Outside Information Copying Document InformationR" RDANumber of DocumentsnCan a paper still be given a score of 5 when the student did not use the minimum number of documents required?oooYes, because it is only one of the criteria on which you base a score. If fewer than the minimum number of documents are used and all other criteria of a 5 are met, then you may have a 5 paper. However, when fewer than the minimum number of documents are used, you often have other criteria that are not being met, and thus you usually end up with a 2, 3, or 4 paper.p" p5/ An example can be found on the August 2009 Global History and Geography DBQ. The Level 5B anchor paper uses 3 documents, instead of the minimum number of 4 documents. This can be found online at: http://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdf P" " mc0EC Minor Errors "60KCan a paper be given a score of 4 or 5 if it contains a minor error or two?LL"If the paper clearly demonstrates that the student understood the task and meets the criteria for a score of 4 or 5 on the test specific rubric, then, the paper can be given that score. A paper does not have to be perfect to reach a score of 5. " 71<An example can be found in the June 2010 United States History and Government DBQ. In this Level 4A anchor paper, the student states that with the passage of the 18th amendment alcohol consumption was made illegal. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdf 6" Z" Z" Z" ZZZ nc05FDOutside InformationHow does  outside information fit into the overall scoring of the essay?JJ"The 4th bullet must not be overemphasized in scoring. A response may receive a 3 without outside information. However, a response with outside information is not automatically a 3.*" -)The quantity and quality of the outside information, the relevance of the outside information to the topic, and how it has been integrated into the essay should be used when determining the score of a response. 6" ZZZsmOften outside information can improve the overall essay when it is used to make historical connections and show analytical thinking. ." ", All papers need to be evaluated based on the rubric. If the paper demonstrates that the student minimally develops the task, is primarily descriptive with weak or isolated information, has a general plan of organization, and includes outside information, the paper is probably more of a 2 than a 3. "," -tnAn example of a level 3 response that uses no outside information can be found on the June 2009 United States History and Government DBQ, Practice Paper B. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdf 2" " pc0G@An example of a level 2 response that uses outside information can be found on the January 2009 United States History and Government DBQ, Level 2A paper. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdf D" ZM" ZZoc0IFCopying Document Information How should you rate a DBQ essay when most of the response is copied directly from the documents and little or no original work is provided? " JK" A distinction needs to be made between paraphrased or copied information that is used to support a specific idea within the essay (an appropriate social studies skill) and the outright copying of information from the documents in an attempted response to the DBQ. 0 "  KGGenerally, when most of the information is copied directly from the documents without adding anything, the response will be rated between a 0 and a 2. 0"  LLMThe revised generic rubrics for the Global and USHG exams address this issue NN Score of 2:  or consists primarily of relevant information copied from the documents Score of 1:  or consists primarily of relevant and irrelevant information copied from the documents Score of 0:  includes only entire documents copied from the test booklet ," MHThe actual score will depend on three things: 1) the degree or extent to which the response has been copied, 2) the selection of what specific content the student chose to copy for inclusion, and 3) how the student integrated the copied material into the response. 0"  NIa Can a paper still be given a score of 5 or 4 when some information is copied from the documents?baxAn example of a level 4 response where the student selectively copied information from documents as well as included good historical outside information to interpret the documents can be found on the January 2009 United States History and Government DBQ, Level 4C paper. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdf 6," L" y@<(What criteria helps define a score of 5?))"KA 5 paper shows evaluation and/or analysis. It requires substantial outside information that is well integrated into the response. It contains historically accurate information. Document information is used to support the thesis of the response and is interwoven with outside information. It is cohesive and thoroughly developed.L" ZLA=A 5 is not a 5 just because it is the best response that exists out of all the responses being graded. A well-written response that contains numerous historical inaccuracies or repeats information over and over again is usually not a level 5 paper. The response needs to meet the criteria established by the rubric. On occasion, a particular essay question may have no papers at level 5." Zlf>What often distinguishes a level 4 paper from a level 3 paper???"cA 4 paper is both analytic and descriptive. It addresses all aspects of the task but may do so unevenly. It requires outside information that is integrated into the response. It contains historically accurate information. Document information is used to support the thesis of the response and is interwoven with outside information. It is well developed. (c" ZZdmgA 3 paper is more descriptive than analytic. It usually contains some outside information but it may not be well integrated. It contains supporting facts and details, but may also have some minor inaccuracies. "" <8>What often distinguishes a level 3 paper from a level 2 paper???"5A level 3 response contains more explanatory statements and specific details than a level 2 response. In a DBQ, the documents are used to support the thesis of the essay. A level 3 response should include outside information, but it may not. However, most solid level 3 papers do include outside information.6" Z6=9A level 2 response tends to have broad general statements. Use of supporting information and information from the documents is limited. Note: While a level 2 paper is not required to include outside information, it may contain some." >:=What most often distinguishes a 0 paper from a level 1 paper?>>"If a response is solely comprised of information copied from the documents and the historical context without any information provided by the student or accurately interpreted by the student, then the response is probably a 0. " B;A level 1 paper is NOT a paper in which the student simply  attempted an answer. It must have some appropriate, original information. *" rb^!Generic Scoring Rubrics by BulletContinuity of Score Points See Appendix E for Thematic Essays: http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf See Appendix F for DBQ Essays: http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf (" " c0?~c0QM Reminder:The rubrics define a set of scores on a continuum between 0 and 5. A standards-based assessment system measures where the student falls on that continuum. @" *%Scoring Related Questions'Issues Related to Scaffolding Questions((" How should you rate an open-ended question when the question asks for two reasons and the student writes two answers on the first line and a third answer on the second line, but one of the answers on the first line is incorrect? *  ?The general rule of thumb is that a rater cannot go searching for a correct answer. Disregarding incorrect information to get to a correct answer is not acceptable. The order in which correct and incorrect answers appear on the lines matters. The first two answers should be rated, even if they are on the same line. &?" @"&!$Issues Related to the DBQ Essay Task%%" How do you rate an essay where the task asks a student to write about two items in the historical context and the response includes discussion of three items? .*_In this situation, only the first two items that appear in the response can be evaluated. A rater cannot search the response to determine which of the three items the student wrote about were strongest. The rater must use the test specific rubric and must focus on information related to the first two items as they appear within the student response.`" Z`keHowever, there may be a situation where a student has appropriately applied information about the third item in a discussion on item one or item two. In this case, if the applied information supports the discussion, it should be considered when rating the overall response. $" +'Other Related Questions UWhy do some of the DBQ or Thematic essays use the phrase and/or within the question? :V9" " "  VSED has been using the phrase and/or on social studies essay questions since 1990. The use of and/or benefits students by broadening the scope of acceptable answers and providing a choice of content within the essay. And/or allows the rater to accept the broadest range of student answers as directed by the content-specific scoring rubric. W" Z""." ""t"  {" EXAMPLE: Discuss the political, social, and/or economic causes for the Russian Revolution. By wording the question this way, a teacher can accept a response that discusses any combination of causes. If the task said  and, the student would have to discuss one correct cause from each category. If the task said  or, the student would have to focus on one type of cause only and a rater could not give credit for information on a different type of cause. LZZ"""yHow do raters know how many examples a student should provide when responding to a particular DBQ or Thematic essay task?zzLook for the  S , the plural. This indicates that two or more examples must be provided. However, in the case of a DBQ, a student must also use a minimum number of documents in his/her response and support the essay with as many facts, examples, and details as he/she can. Thus, having more examples usually improves the essay. Collectively, the DBQ documents will always provide multiple examples. @" ZZ jd(If only one cause or impact is asked for, it will usually be phrased  a or  one . Example: a cause of& or one cause of& In general, teachers should teach their students to use as many examples as they can. It adds detail to the essay. It helps show the student s knowledge. " F fbOther Scoring Notes` In Global History, can we rate essay responses in which students wrote about the United States?aaMany topics in Global and U.S. History are intertwined. (The Cold War, The Middle East, Vietnam War, etc.) Not all essays will automatically exclude the use of information on the U.S. as part of an acceptable answer. An example of this can be found on the January 2011 Global History and Geography thematic essay. This can be found online at: http://www.nysedregents.org/GlobalHistoryGeography/20110128-gh-rg1.pdf *" ZZwpTools required for scoringDirections for Administering Regents Examinations www.p12.nysed.gov/assessment/hsgen H & G Information Booklet for Scoring Regents Examinations in Global History and Geography and United States History and Government www.p12.nysed.gov/assessment/hsgen School Administrator s Manual, Secondary Level Examinations http://www.p12.nysed.gov/assessment/sam/secondary/home.html .8" P>" Pv^c02T_c0c09txqGlobal History and Geography and United States History and Government rating guides for specific test administrations available online (" " yrTest manuals should be reviewed by all administrators, teachers, and staff administering and/or scoring a 鶹ýӳ State examination. Anyone who is involved in the scoring of a 鶹ýӳ State examination must complete the training as specified in the test manuals and rating guides." ZReminder2Teachers are allowed to proctor their own students, unless the district has a different policy. Teachers may not score their own students answer papers." y Training All training should be based on the protocols established by 鶹ýӳ as found in the H & G Information Booklet for Scoring Regents Examinations in Global History and Geography and United States History and Government as well as in the test specific rating guides." zScorers are encouraged to ask questions that relate to understanding the scoring guidelines and questionable papers, or scaffolding responses, but philosophical differences, personal preferences and biases must be set aside during the scoring process." {The goal of scoring is to be sure the scoring criteria in the rubrics are accurately and consistently applied to each student response being rated across 鶹ýӳ State." |-Questions about scoring social studies exams.."Contact the Associates in Social Studies in the Office of State Assessment For Global History and Geography: Donna Merlau, dmerlau@mail.nysed.gov For United States History and Government: Greg Wilsey, gwilsey@mail.nysed.gov Phone: 518-474-0556 rn" '" *" %" " " Pu2c0}c0icYour input is importantFeedback on State examinations is very important to the test development process. This can be done by completing our online Teacher Evaluation Forms at: http://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfm B" @" " Ac0r_A# _1( #ssessment/sam/secondary/home.htmlvhttp://www.p12.nysed.gov/assessment/sam/secondary/home.html,dmerlau@mail.nysed.gov:mailto:dmerlau@mail.nysed.gov,gwilsey@mail.nysed.gov:mailto:gwilsey@mail.nysed.gov4http://www.nysedregents.org/GlobalHistoryGeography/20110128-gh-rg1.pdfhttp://www.nysedregents.org/GlobalHistoryGeography/20110128-gh-rg1.pdfDhttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfF/ 0|DArialgsRomanPP`:ؖF0ؖDVerdanasRomanPP`:ؖF0ؖ" DTimes New RomanPP`:ؖF0ؖ0DWingdingsRomanPP`:ؖF0ؖ@DTahomagsRomanPP`:ؖF0ؖ" A .  @n?" dd@  @@``   X= 21   () 5  >>    :<=C       tuv|}             ?r$kW7{y3Ti$b$9]+?Pe~.*)$i 0AA@8gXhYi mS ʚ;l8ʚ;g4<d<dPF0ppp@ <4dddd@0P<:<4BdBd@0P<: <4!d!d(A0P0___PPT10 ___PPT9@}~u[/01)mn@KGLHI<=fg89:;^M *e Tdpqryz{|c?  %bh The 鶹ýӳ State Education Department Demystifying the Holistic Scoring Process in Social Studies: Hi$($;lRegents Examinations in Global History and Geography and United States History and Government Spring 2013 "mZl{tHolistic Scoring OverviewWhen scoring holistically Read thoroughly, yet quickly to gain an impression of the entire response Read the entire response before determining a score, and then promptly assign a score J" " }Scores must be determined based on the holistic application of the content-specific rubric provided by 鶹ýӳ. The anchor and practice papers serve as examples of what a student response at a particular score level should look like. The content-specific rubric is not a checklist. " Z~ Important points to keep in mind!!"The quality of handwriting, the use of cursive or printing, margins, editing marks, cross-outs, and overall neatness are not part of the scoring criteria, unless the response is completely illegible and nothing can be read. " |u(It is important for each reader to set aside his/her biases in order to keep the scoring as consistent and as fair to each student as possible. You are scoring a State exam, not grading a classroom assignment, therefore, personal classroom grading practices should be set aside while scoring. )" Z)_[@The Six Primary Scoring Criteria for the Commencement-Level DBQAAThe extent to which students develop the task utilize higher level thinking skills use the information provided in the documents incorporate relevant outside information in the DBQ essay support ideas with facts, examples, and details organize and develop a social studies essay*" " 7No one bullet can hold a particular score point hostage88"QNumber of Documents Minor Errors Outside Information Copying Document InformationR" RDANumber of DocumentsnCan a paper still be given a score of 5 when the student did not use the minimum number of documents required?oooYes, because it is only one of the criteria on which you base a score. If fewer than the minimum number of documents are used and all other criteria of a 5 are met, then you may have a 5 paper. However, when fewer than the minimum number of documents are used, you often have other criteria that are not being met, and thus you usually end up with a 2, 3, or 4 paper.p" p5/ An example can be found on the August 2009 Global History and Geography DBQ. The Level 5B anchor paper uses 3 documents, instead of the minimum number of 4 documents. This can be found online at: http://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdf P" " mc0EC Minor Errors "60KCan a paper be given a score of 4 or 5 if it contains a minor error or two?LL"If the paper clearly demonstrates that the student understood the task and meets the criteria for a score of 4 or 5 on the test specific rubric, then, the paper can be given that score. A paper does not have to be perfect to reach a score of 5. " 71<An example can be found in the June 2010 United States History and Government DBQ. In this Level 4A anchor paper, the student states that with the passage of the 18th amendment alcohol consumption was made illegal. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdf 6" Z" Z" Z" ZZZ nc05FDOutside InformationHow does  outside information fit into the overall scoring of the essay?JJ"The 4th bullet must not be overemphasized in scoring. A response may receive a 3 without outside information. However, a response with outside information is not automatically a 3.*" -)The quantity and quality of the outside information, the relevance of the outside information to the topic, and how it has been integrated into the essay should be used when determining the score of a response. 6" ZZZsmOften outside information can improve the overall essay when it is used to make historical connections and show analytical thinking. ." ", All papers need to be evaluated based on the rubric. If the paper demonstrates that the student minimally develops the task, is primarily descriptive with weak or isolated information, has a general plan of organization, and includes outside information, the paper is probably more of a 2 than a 3. "," -tnAn example of a level 3 response that uses no outside information can be found on the June 2009 United States History and Government DBQ, Practice Paper B. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdf 2" " pc0G@An example of a level 2 response that uses outside information can be found on the January 2009 United States History and Government DBQ, Level 2A paper. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdf D" ZM" ZZoc0IFCopying Document Information How should you rate a DBQ essay when most of the response is copied directly from the documents and little or no original work is provided? " JK" A distinction needs to be made between paraphrased or copied information that is used to support a specific idea within the essay (an appropriate social studies skill) and the outright copying of information from the documents in an attempted response to the DBQ. 0 "  KGGenerally, when most of the information is copied directly from the documents without adding anything, the response will be rated between a 0 and a 2. 0"  LLMThe revised generic rubrics for the Global and USHG exams address this issue NN Score of 2:  or consists primarily of relevant information copied from the documents Score of 1:  or consists primarily of relevant and irrelevant information copied from the documents Score of 0:  includes only entire documents copied from the test booklet ," MHThe actual score will depend on three things: 1) the degree or extent to which the response has been copied, 2) the selection of what specific content the student chose to copy for inclusion, and 3) how the student integrated the copied material into the response. 0"  NIa Can a paper still be given a score of 5 or 4 when some information is copied from the documents?baxAn example of a level 4 response where the student selectively copied information from documents as well as included good historical outside information to interpret the documents can be found on the January 2009 United States History and Government DBQ, Level 4C paper. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdf 6," L" y-Jc0-w@<(What criteria helps define a score of 5?))"KA 5 paper shows evaluation and/or analysis. It requires substantial outside information that is well integrated into the response. It contains historically accurate information. Document information is used to support the thesis of the response and is interwoven with outside information. It is cohesive and thoroughly developed.L" ZLA=A 5 is not a 5 just because it is the best response that exists out of all the responses being graded. A well-written response that contains numerous historical inaccuracies or repeats information over and over again is usually not a level 5 paper. The response needs to meet the criteria established by the rubric. On occasion, a particular essay question may have no papers at level 5." Zlf>What often distinguishes a level 4 paper from a level 3 paper???"cA 4 paper is both analytic and descriptive. It addresses all aspects of the task but may do so unevenly. It requires outside information that is integrated into the response. It contains historically accurate information. Document information is used to support the thesis of the response and is interwoven with outside information. It is well developed. (c" ZZdmgA 3 paper is more descriptive than analytic. It usually contains some outside information but it may not be well integrated. It contains supporting facts and details, but may also have some minor inaccuracies. "" <8>What often distinguishes a level 3 paper from a level 2 paper???"5A level 3 response contains more explanatory statements and specific details than a level 2 response. In a DBQ, the documents are used to support the thesis of the essay. A level 3 response should include outside information, but it may not. However, most solid level 3 papers do include outside information.6" Z6=9A level 2 response tends to have broad general statements. Use of supporting information and information from the documents is limited. Note: While a level 2 paper is not required to include outside information, it may contain some." >:=What most often distinguishes a 0 paper from a level 1 paper?>>"If a response is solely comprised of information copied from the documents and the historical context without any information provided by the student or accurately interpreted by the student, then the response is probably a 0. " B;A level 1 paper is NOT a paper in which the student simply  attempted an answer. It must have some appropriate, original information. *" rb^!Generic Scoring Rubrics by BulletContinuity of Score Points See Appendix E for Thematic Essays: http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf See Appendix F for DBQ Essays: http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf (" " c0?~c0QM Reminder:The rubrics define a set of scores on a continuum between 0 and 5. A standards-based assessment system measures where the student falls on that continuum. @" *%Scoring Related Questions'Issues Related to Scaffolding Questions((" How should you rate an open-ended question when the question asks for two reasons and the student writes two answers on the first line and a third answer on the second line, but one of the answers on the first line is incorrect? *  ?The general rule of thumb is that a rater cannot go searching for a correct answer. Disregarding incorrect information to get to a correct answer is not acceptable. The order in which correct and incorrect answers appear on the lines matters. The first two answers should be rated, even if they are on the same line. &?" @"&!$Issues Related to the DBQ Essay Task%%" How do you rate an essay where the task asks a student to write about two items in the historical context and the response includes discussion of three items? .*_In this situation, only the first two items that appear in the response can be evaluated. A rater cannot search the response to determine which of the three items the student wrote about were strongest. The rater must use the test specific rubric and must focus on information related to the first two items as they appear within the student response.`" Z`keHowever, there may be a situation where a student has appropriately applied information about the third item in a discussion on item one or item two. In this case, if the applied information supports the discussion, it should be considered when rating the overall response. $" +'Other Related Questions UWhy do some of the DBQ or Thematic essays use the phrase and/or within the question? :V9" " "  VSED has been using the phrase and/or on social studies essay questions since 1990. The use of and/or benefits students by broadening the scope of acceptable answers and providing a choice of content within the essay. And/or allows the rater to accept the broadest range of student answers as directed by the content-specific scoring rubric. W" Z""." ""t"  {" EXAMPLE: Discuss the political, social, and/or economic causes for the Russian Revolution. By wording the question this way, a teacher can accept a response that discusses any combination of causes. If the task said  and, the student would have to discuss one correct cause from each category. If the task said  or, the student would have to focus on one type of cause only and a rater could not give credit for information on a different type of cause. LZZ"""yHow do raters know how many examples a student should provide when responding to a particular DBQ or Thematic essay task?zzLook for the  S , the plural. This indicates that two or more examples must be provided. However, in the case of a DBQ, a student must also use a minimum number of documents in his/her response and support the essay with as many facts, examples, and details as he/she can. Thus, having more examples usually improves the essay. Collectively, the DBQ documents will always provide multiple examples. @" ZZ jd(If only one cause or impact is asked for, it will usually be phrased  a or  one . Example: a cause of& or one cause of& In general, teachers should teach their students to use as many examples as they can. It adds detail to the essay. It helps show the student s knowledge. " F fbOther Scoring Notes` In Global History, can we rate essay responses in which students wrote about the United States?aaMany topics in Global and U.S. History are intertwined. (The Cold War, The Middle East, Vietnam War, etc.) Not all essays will automatically exclude the use of information on the U.S. as part of an acceptable answer. An example of this can be found on the January 2011 Global History and Geography thematic essay. This can be found online at: http://www.nysedregents.org/GlobalHistoryGeography/20110128-gh-rg1.pdf *" ZZc0WwpTools required for scoringDirections for Administering Regents Examinations www.p12.nysed.gov/assessment/hsgen H & G Information Booklet for Scoring Regents Examinations in Global History and Geography and United States History and Government www.p12.nysed.gov/assessment/hsgen School Administrator s Manual, Secondary Level Examinations http://www.p12.nysed.gov/assessment/sam/secondary/home.html .8" P>" Pv^c02T_c0c09txqGlobal History and Geography and United States History and Government rating guides for specific test administrations available online (" " yrTest manuals should be reviewed by all administrators, teachers, and staff administering and/or scoring a 鶹ýӳ State examination. Anyone who is involved in the scoring of a 鶹ýӳ State examination must complete the training as specified in the test manuals and rating guides." ZReminder2Teachers are allowed to proctor their own students, unless the district has a different policy. Teachers may not score their own students answer papers." y Training All training should be based on the protocols established by 鶹ýӳ as found in the H & G Information Booklet for Scoring Regents Examinations in Global History and Geography and United States History and Government as well as in the test specific rating guides." zScorers are encouraged to ask questions that relate to understanding the scoring guidelines and questionable papers, or scaffolding responses, but philosophical differences, personal preferences and biases must be set aside during the scoring process." {The goal of scoring is to be sure the scoring criteria in the rubrics are accurately and consistently applied to each student response being rated across 鶹ýӳ State." |-Questions about scoring social studies exams.."Contact the Associates in Social Studies in the Office of State Assessment For Global History and Geography: Donna Merlau, dmerlau@mail.nysed.gov For United States History and Government: Greg Wilsey, gwilsey@mail.nysed.gov Phone: 518-474-0556 rn" '" *" %" " " Pu2c0}c0icYour input is importantFeedback on State examinations is very important to the test development process. This can be done by completing our online Teacher Evaluation Forms at: http://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfm B" @" " Ac0 X0  *(  r"  S d,`   x  c $TPe  H  0޽h ? 3fffff3̙3f̙___PPT10i.:Py+D=' 7= @B + X0N0 <(  ~  s *߷`   ~  s *Pe  H  0޽h ? f3y___PPT10Y+D=' 7= @B +rRqNnI# s1( # f thttp://www.nysedregents.org/testing/socstre/ushgrg.605.pdfthttp://www.nysedregents.org/testing/socstre/ushgrg.605.pdf#4emscreminfo@mail.nysed.govBmailto:emscoryGov/Archive/20090129scoringkeyIII.pdfKhttp://www.nysedregents.org/GlobalHistoryGeography/611/20110615-gh-rg1.pdfreminfo@mail.nysed.gov^Dwww.p12.nysed.gov/assessment/hsgenRhttp://www.p12.nysed.gov/assessment/hsgen_Dwww.p12.nysed.gov/assessment/hsgenRhttp://www.p12.nysed.gov/assessment/hsgenlmhttp://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdfhttp://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdfnhttp://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdfDohttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfDphttp://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf xhttp://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfmxhttp://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfmvhttp://www.p12.nysed.gov/assessment/sam/secondary/home.htmlvhttp://www.p12.nysed.gov/assessment/sam/secondary/home.html,dmerlau@mail.nysed.gov:mailto:dmerlau@mail.nysed.gov,gwilsey@mail.nysed.gov:mailto:gwilsey@mail.nysed.govDhttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfDhttp://www.nysedregents.org/GlobalHistoryGeography/611/20110615-gh-rg1.pdfhttp://www.nysedregents.org/GlobalHistoryGeography/611/20110615-gh-rg1.pdfF/ 0|DArialgsRomanPPhuؖF0ؖ"DVerdanasRomanPPhuؖF0ؖ" DTimes New RomanPPhuؖF0ؖ0DWingdingsRomanPPhuؖF0ؖ@DTahomagsRomanPPhuؖF0ؖ" A .  @n?" dd@  @@``   X= 21   () 5  >>    :<=C       tuv|}             ?r$kW7{y3Ti$b$9]+?Pe~.*)$i 0AA@8gXhYK mS ʚ;l8ʚ;g4<d<d`ƘF0ppp@ <4dddd@0PDu<4BdBd@0PDu <4!d!d(A0P0___PPT10 ___PPT9jB}~u[/01)mn@KGLHI<=fg89:;^M *e Tdpqryz{|c?  %bh The 鶹ýӳ State Education Department Demystifying the Holistic Scoring Process in Social Studies: Hi$($;lRegents Examinations in Global History and Geography and United States History and Government Spring 2013 "mZl{tHolistic Scoring OverviewWhen scoring holistically Read thoroughly, yet quickly to gain an impression of the entire response Read the entire response before determining a score, and then promptly assign a score J" " }Scores must be determined based on the holistic application of the content-specific rubric provided by 鶹ýӳ. The anchor and practice papers serve as examples of what a student response at a particular score level should look like. The content-specific rubric is not a checklist. " Z~ Important points to keep in mind!!"The quality of handwriting, the use of cursive or printing, margins, editing marks, cross-outs, and overall neatness are not part of the scoring criteria, unless the response is completely illegible and nothing can be read. " |u(It is important for each reader to set aside his/her biases in order to keep the scoring as consistent and as fair to each student as possible. You are scoring a State exam, not grading a classroom assignment, therefore, personal classroom grading practices should be set aside while scoring. )" Z)_[@The Six Primary Scoring Criteria for the Commencement-Level DBQAAThe extent to which students develop the task utilize higher level thinking skills use the information provided in the documents incorporate relevant outside information in the DBQ essay support ideas with facts, examples, and details organize and develop a social studies essay*" " 7No one bullet can hold a particular score point hostage88"QNumber of Documents Minor Errors Outside Information Copying Document InformationR" RDANumber of DocumentsnCan a paper still be given a score of 5 when the student did not use the minimum number of documents required?oooYes, because it is only one of the criteria on which you base a score. If fewer than the minimum number of documents are used and all other criteria of a 5 are met, then you may have a 5 paper. However, when fewer than the minimum number of documents are used, you often have other criteria that are not being met, and thus you usually end up with a 2, 3, or 4 paper.p" p5/ An example can be found on the August 2009 Global History and Geography DBQ. The Level 5B anchor paper uses 3 documents, instead of the minimum number of 4 documents. This can be found online at: http://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdf P" " mc0EC Minor Errors "60KCan a paper be given a score of 4 or 5 if it contains a minor error or two?LL"If the paper clearly demonstrates that the student understood the task and meets the criteria for a score of 4 or 5 on the test specific rubric, then, the paper can be given that score. A paper does not have to be perfect to reach a score of 5. " 71<An example can be found in the June 2010 United States History and Government DBQ. In this Level 4A anchor paper, the student states that with the passage of the 18th amendment alcohol consumption was made illegal. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdf 6" Z" Z" Z" ZZZ nc05FDOutside InformationHow does  outside information fit into the overall scoring of the essay?JJ"The 4th bullet must not be overemphasized in scoring. A response may receive a 3 without outside information. However, a response with outside information is not automatically a 3.*" -)The quantity and quality of the outside information, the relevance of the outside information to the topic, and how it has been integrated into the essay should be used when determining the score of a response. 6" ZZZsmOften outside information can improve the overall essay when it is used to make historical connections and show analytical thinking. ." ", All papers need to be evaluated based on the rubric. If the paper demonstrates that the student minimally develops the task, is primarily descriptive with weak or isolated information, has a general plan of organization, and includes outside information, the paper is probably more of a 2 than a 3. "," -tnAn example of a level 3 response that uses no outside information can be found on the June 2009 United States History and Government DBQ, Practice Paper B. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdf 2" " pc0G@An example of a level 2 response that uses outside information can be found on the January 2009 United States History and Government DBQ, Level 2A paper. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdf D" ZM" ZZoc0IFCopying Document Information How should you rate a DBQ essay when most of the response is copied directly from the documents and little or no original work is provided? " JK" A distinction needs to be made between paraphrased or copied information that is used to support a specific idea within the essay (an appropriate social studies skill) and the outright copying of information from the documents in an attempted response to the DBQ. 0 "  KGGenerally, when most of the information is copied directly from the documents without adding anything, the response will be rated between a 0 and a 2. 0"  LLMThe revised generic rubrics for the Global and USHG exams address this issue NN Score of 2:  or consists primarily of relevant information copied from the documents Score of 1:  or consists primarily of relevant and irrelevant information copied from the documents Score of 0:  includes only entire documents copied from the test booklet ," MHThe actual score will depend on three things: 1) the degree or extent to which the response has been copied, 2) the selection of what specific content the student chose to copy for inclusion, and 3) how the student integrated the copied material into the response. 0"  NIa Can a paper still be given a score of 5 or 4 when some information is copied from the documents?baxAn example of a level 4 response where the student selectively copied information from documents as well as included good historical outside information to interpret the documents can be found on the January 2009 United States History and Government DBQ, Level 4C paper. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdf 6," L" yc0-w@<(What criteria helps define a score of 5?))"KA 5 paper shows evaluation and/or analysis. It requires substantial outside information that is well integrated into the response. It contains historically accurate information. Document information is used to support the thesis of the response and is interwoven with outside information. It is cohesive and thoroughly developed.L" ZLA=A 5 is not a 5 just because it is the best response that exists out of all the responses being graded. A well-written response that contains numerous historical inaccuracies or repeats information over and over again is usually not a level 5 paper. The response needs to meet the criteria established by the rubric. On occasion, a particular essay question may have no papers at level 5." Zlf>What often distinguishes a level 4 paper from a level 3 paper???"cA 4 paper is both analytic and descriptive. It addresses all aspects of the task but may do so unevenly. It requires outside information that is integrated into the response. It contains historically accurate information. Document information is used to support the thesis of the response and is interwoven with outside information. It is well developed. (c" ZZdmgA 3 paper is more descriptive than analytic. It usually contains some outside information but it may not be well integrated. It contains supporting facts and details, but may also have some minor inaccuracies. "" <8>What often distinguishes a level 3 paper from a level 2 paper???"5A level 3 response contains more explanatory statements and specific details than a level 2 response. In a DBQ, the documents are used to support the thesis of the essay. A level 3 response should include outside information, but it may not. However, most solid level 3 papers do include outside information.6" Z6=9A level 2 response tends to have broad general statements. Use of supporting information and information from the documents is limited. Note: While a level 2 paper is not required to include outside information, it may contain some." >:=What most often distinguishes a 0 paper from a level 1 paper?>>"If a response is solely comprised of information copied from the documents and the historical context without any information provided by the student or accurately interpreted by the student, then the response is probably a 0. " B;A level 1 paper is NOT a paper in which the student simply  attempted an answer. It must have some appropriate, original information. *" rb^!Generic Scoring Rubrics by BulletContinuity of Score Points See Appendix E for Thematic Essays: http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf See Appendix F for DBQ Essays: http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf (" " c0?~c0QM Reminder:The rubrics define a set of scores on a continuum between 0 and 5. A standards-based assessment system measures where the student falls on that continuum. @" *%Scoring Related Questions'Issues Related to Scaffolding Questions((" How should you rate an open-ended question when the question asks for two reasons and the student writes two answers on the first line and a third answer on the second line, but one of the answers on the first line is incorrect? *  ?The general rule of thumb is that a rater cannot go searching for a correct answer. Disregarding incorrect information to get to a correct answer is not acceptable. The order in which correct and incorrect answers appear on the lines matters. The first two answers should be rated, even if they are on the same line. &?" @"&!$Issues Related to the DBQ Essay Task%%" How do you rate an essay where the task asks a student to write about two items in the historical context and the response includes discussion of three items? .*_In this situation, only the first two items that appear in the response can be evaluated. A rater cannot search the response to determine which of the three items the student wrote about were strongest. The rater must use the test specific rubric and must focus on information related to the first two items as they appear within the student response.`" Z`keHowever, there may be a situation where a student has appropriately applied information about the third item in a discussion on item one or item two. In this case, if the applied information supports the discussion, it should be considered when rating the overall response. $" +'Other Related Questions UWhy do some of the DBQ or Thematic essays use the phrase and/or within the question? :V9" " "  VSED has been using the phrase and/or on social studies essay questions since 1990. The use of and/or benefits students by broadening the scope of acceptable answers and providing a choice of content within the essay. And/or allows the rater to accept the broadest range of student answers as directed by the content-specific scoring rubric. W" Z""." ""t"  {" EXAMPLE: Discuss the political, social, and/or economic causes for the Russian Revolution. By wording the question this way, a teacher can accept a response that discusses any combination of causes. If the task said  and, the student would have to discuss one correct cause from each category. If the task said  or, the student would have to focus on one type of cause only and a rater could not give credit for information on a different type of cause. LZZ"""yHow do raters know how many examples a student should provide when responding to a particular DBQ or Thematic essay task?zzLook for the  S , the plural. This indicates that two or more examples must be provided. However, in the case of a DBQ, a student must also use a minimum number of documents in his/her response and support the essay with as many facts, examples, and details as he/she can. Thus, having more examples usually improves the essay. Collectively, the DBQ documents will always provide multiple examples. @" ZZ jd(If only one cause or impact is asked for, it will usually be phrased  a or  one . Example: a cause of& or one cause of& In general, teachers should teach their students to use as many examples as they can. It adds detail to the essay. It helps show the student s knowledge. " F fbOther Scoring Notes` In Global History, can we rate essay responses in which students wrote about the United States?aaMany topics in Global and U.S. History are intertwined. (The Cold War, The Middle East, Vietnam War, etc.) Not all essays will automatically exclude the use of information on the U.S. as part of an acceptable answer. An example of this can be found on the June 2011 Global History and Geography thematic essay. This can be found online at: http://www.nysedregents.org/GlobalHistoryGeography/611/20110615-gh-rg1.pdf. >" PPTK6PJc0TwpTools required for scoringDirections for Administering Regents Examinations www.p12.nysed.gov/assessment/hsgen H & G Information Booklet for Scoring Regents Examinations in Global History and Geography and United States History and Government www.p12.nysed.gov/assessment/hsgen School Administrator s Manual, Secondary Level Examinations http://www.p12.nysed.gov/assessment/sam/secondary/home.html .8" P>" Pv^c02T_c0c09txqGlobal History and Geography and United States History and Government rating guides for specific test administrations available online (" " yrTest manuals should be reviewed by all administrators, teachers, and staff administering and/or scoring a 鶹ýӳ State examination. Anyone who is involved in the scoring of a 鶹ýӳ State examination must complete the training as specified in the test manuals and rating guides." ZReminder2Teachers are allowed to proctor their own students, unless the district has a different policy. Teachers may not score their own students answer papers." y Training All training should be based on the protocols established by 鶹ýӳ as found in the H & G Information Booklet for Scoring Regents Examinations in Global History and Geography and United States History and Government as well as in the test specific rating guides." zScorers are encouraged to ask questions that relate to understanding the scoring guidelines and questionable papers, or scaffolding responses, but philosophical differences, personal preferences and biases must be set aside during the scoring process." {The goal of scoring is to be sure the scoring criteria in the rubrics are accurately and consistently applied to each student response being rated across 鶹ýӳ State." |-Questions about scoring social studies exams.."Contact the Associates in Social Studies in the Office of State Assessment For Global History and Geography: Donna Merlau, dmerlau@mail.nysed.gov For United States History and Government: Greg Wilsey, gwilsey@mail.nysed.gov Phone: 518-474-0556 rn" '" *" %" " " Pu2c0}c0icYour input is importantFeedback on State examinations is very important to the test development process. This can be done by completing our online Teacher Evaluation Forms at: http://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfm B" @" " Ac0 X0N0 <(  ~  s *@U6`  6 ~  s *N6Pe 6 H  0޽h ? f3y___PPT10Y+D=' = @B +r;t_# t1ވ( # f thttp://www.nysedregents.org/testing/socstre/ushgrg.605.pdfthttp://www.nysedregents.org/testing/socstre/ushgrg.605.pdf#4emscreminfo@mail.nysed.govBmailto:e      !"#$%&'()*+,-./0123456789:;<=>?@ABmscreminfo@mail.nysed.gov^Dwww.p12.nysed.gov/assessment/hsgenRhttp://www.p12.nysed.gov/assessment/hsgen_Dwww.p12.nysed.gov/assessment/hsgenRhttp://www.p12.nysed.gov/assessment/hsgenlmhttp://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdfhttp://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdfnhttp://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdfDohttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfDphttp://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf~http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf xhttp://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfmxhttp://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfmvhttp://www.p12.nysed.gov/assessment/sam/secondary/home.htmlvhttp://www.p12.nysed.gov/assessment/sam/secondary/home.html,dmerlau@mail.nysed.gov:mailto:dmerlau@mail.nysed.gov,gwilsey@mail.nysed.gov:mailto:gwilsey@mail.nysed.govDhttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfhttp://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdfDhttp://www.nysedregents.org/GlobalHistoryGeography/611/20110615-gh-rg1.pdfhttp://www.nysedregents.org/GlobalHistoryGeography/611/20110615-gh-rg1.pdfF/ 0|DArialgsRomanPPhuؖF0ؖ"DVerdanasRomanPPhuؖF0ؖ" DTimes New RomanPPhuؖF0ؖ0DWingdingsRomanPPhuؖF0ؖ@DTahomagsRomanPPhuؖF0ؖ" A .  @n?" dd@  @@``   X= 21   () 5  >>    :<=C       tuv|}             ?r$kW7{y3Ti$b$9]+?Pe~.*)$i 0AA@8gXhYK mS ʚ;l8ʚ;g4<d<d`ƘF0ppp@ <4dddd@0PDu<4BdBd@0PDu <4!d!d(A0P0___PPT10 ___PPT9jB}~u[/01)mn@KGLHI<=fg89:;^M *e Tdpqryz{|c?  %bh The 鶹ýӳ State Education Department Demystifying the Holistic Scoring Process in Social Studies: Hi$($;lRegents Examinations in Global History and Geography and United States History and Government Spring 2013 "mZl{tHolistic Scoring OverviewWhen scoring holistically Read thoroughly, yet quickly to gain an impression of the entire response Read the entire response before determining a score, and then promptly assign a score J" " }Scores must be determined based on the holistic application of the content-specific rubric provided by 鶹ýӳ. The anchor and practice papers serve as examples of what a student response at a particular score level should look like. The content-specific rubric is not a checklist. " Z~ Important points to keep in mind!!"The quality of handwriting, the use of cursive or printing, margins, editing marks, cross-outs, and overall neatness are not part of the scoring criteria, unless the response is completely illegible and nothing can be read. " |u(It is important for each reader to set aside his/her biases in order to keep the scoring as consistent and as fair to each student as possible. You are scoring a State exam, not grading a classroom assignment, therefore, personal classroom grading practices should be set aside while scoring. )" Z)_[@The Six Primary Scoring Criteria for the Commencement-Level DBQAAThe extent to which students develop the task utilize higher level thinking skills use the information provided in the documents incorporate relevant outside information in the DBQ essay support ideas with facts, examples, and details organize and develop a social studies essay*" " 7No one bullet can hold a particular score point hostage88"QNumber of Documents Minor Errors Outside Information Copying Document InformationR" RDANumber of DocumentsnCan a paper still be given a score of 5 when the student did not use the minimum number of documents required?oooYes, because it is only one of the criteria on which you base a score. If fewer than the minimum number of documents are used and all other criteria of a 5 are met, then you may have a 5 paper. However, when fewer than the minimum number of documents are used, you often have other criteria that are not being met, and thus you usually end up with a 2, 3, or 4 paper.p" p5/ An example can be found on the August 2009 Global History and Geography DBQ. The Level 5B anchor paper uses 3 documents, instead of the minimum number of 4 documents. This can be found online at: http://www.nysedregents.org/GlobalHistoryGeography/Archive/20090812scoringkeyIII.pdf P" " mc0EC Minor Errors "60KCan a paper be given a score of 4 or 5 if it contains a minor error or two?LL"If the paper clearly demonstrates that the student understood the task and meets the criteria for a score of 4 or 5 on the test specific rubric, then, the paper can be given that score. A paper does not have to be perfect to reach a score of 5. " 71<An example can be found in the June 2010 United States History and Government DBQ. In this Level 4A anchor paper, the student states that with the passage of the 18th amendment alcohol consumption was made illegal. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20100618rg2.pdf 6" Z" Z" Z" ZZZ nc05FDOutside InformationHow does  outside information fit into the overall scoring of the essay?JJ"The 4th bullet must not be overemphasized in scoring. A response may receive a 3 without outside information. However, a response with outside information is not automatically a 3.*" -)The quantity and quality of the outside information, the relevance of the outside information to the topic, and how it has been integrated into the essay should be used when determining the score of a response. 6" ZZZsmOften outside information can improve the overall essay when it is used to make historical connections and show analytical thinking. ." ", All papers need to be evaluated based on the rubric. If the paper demonstrates that the student minimally develops the task, is primarily descriptive with weak or isolated information, has a general plan of organization, and includes outside information, the paper is probably more of a 2 than a 3. "," -tnAn example of a level 3 response that uses no outside information can be found on the June 2009 United States History and Government DBQ, Practice Paper B. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090616scoringkeyIII.pdf 2" " pc0G@An example of a level 2 response that uses outside information can be found on the January 2009 United States History and Government DBQ, Level 2A paper. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdf D" ZM" ZZoc0IFCopying Document Information How should you rate a DBQ essay when most of the response is copied directly from the documents and little or no original work is provided? " JK" A distinction needs to be made between paraphrased or copied information that is used to support a specific idea within the essay (an appropriate social studies skill) and the outright copying of information from the documents in an attempted response to the DBQ. 0 "  KGGenerally, when most of the information is copied directly from the documents without adding anything, the response will be rated between a 0 and a 2. 0"  LLMThe revised generic rubrics for the Global and USHG exams address this issue NN Score of 2:  or consists primarily of relevant information copied from the documents Score of 1:  or consists primarily of relevant and irrelevant information copied from the documents Score of 0:  includes only entire documents copied from the test booklet ," MHThe actual score will depend on three things: 1) the degree or extent to which the response has been copied, 2) the selection of what specific content the student chose to copy for inclusion, and 3) how the student integrated the copied material into the response. 0"  NIa Can a paper still be given a score of 5 or 4 when some information is copied from the documents?baxAn example of a level 4 response where the student selectively copied information from documents as well as included good historical outside information to interpret the documents can be found on the January 2009 United States History and Government DBQ, Level 4C paper. This can be found online at: http://www.nysedregents.org/USHistoryGov/Archive/20090129scoringkeyIII.pdf 6," L" yc0-w@<(What criteria helps define a score of 5?))"KA 5 paper shows evaluation and/or analysis. It requires substantial outside information that is well integrated into the response. It contains historically accurate information. Document information is used to support the thesis of the response and is interwoven with outside information. It is cohesive and thoroughly developed.L" ZLA=A 5 is not a 5 just because it is the best response that exists out of all the responses being graded. A well-written response that contains numerous historical inaccuracies or repeats information over and over again is usually not a level 5 paper. The response needs to meet the criteria established by the rubric. On occasion, a particular essay question may have no papers at level 5." Zlf>What often distinguishes a level 4 paper from a level 3 paper???"cA 4 paper is both analytic and descriptive. It addresses all aspects of the task but may do so unevenly. It requires outside information that is integrated into the response. It contains historically accurate information. Document information is used to support the thesis of the response and is interwoven with outside information. It is well developed. (c" ZZdmgA 3 paper is more descriptive than analytic. It usually contains some outside information but it may not be well integrated. It contains supporting facts and details, but may also have some minor inaccuracies. "" <8>What often distinguishes a level 3 paper from a level 2 paper???"5A level 3 response contains more explanatory statements and specific details than a level 2 response. In a DBQ, the documents are used to support the thesis of the essay. A level 3 response should include outside information, but it may not. However, most solid level 3 papers do include outside information.6" Z6=9A level 2 response tends to have broad general statements. Use of supporting information and information from the documents is limited. Note: While a level 2 paper is not required to include outside information, it may contain some." >:=What most often distinguishes a 0 paper from a level 1 paper?>>"If a response is solely comprised of information copied from the documents and the historical context without any information provided by the student or accurately interpreted by the student, then the response is probably a 0. " B;A level 1 paper is NOT a paper in which the student simply  attempted an answer. It must have some appropriate, original information. *" rb^!Generic Scoring Rubrics by BulletContinuity of Score Points See Appendix E for Thematic Essays: http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-e.pdf See Appendix F for DBQ Essays: http://www.p12.nysed.gov/assessment/ss/hs/rubrics/rubrics-f.pdf (" " c0?~c0QM Reminder:The rubrics define a set of scores on a continuum between 0 and 5. A standards-based assessment system measures where the student falls on that continuum. @" *%Scoring Related Questions'Issues Related to Scaffolding Questions((" How should you rate an open-ended question when the question asks for two reasons and the student writes two answers on the first line and a third answer on the second line, but one of the answers on the first line is incorrect? *  ?The general rule of thumb is that a rater cannot go searching for a correct answer. Disregarding incorrect information to get to a correct answer is not acceptable. The order in which correct and incorrect answers appear on the lines matters. The first two answers should be rated, even if they are on the same line. &?" @"&!$Issues Related to the DBQ Essay Task%%" How do you rate an essay where the task asks a student to write about two items in the historical context and the response includes discussion of three items? .*_In this situation, only the first two items that appear in the response can be evaluated. A rater cannot search the response to determine which of the three items the student wrote about were strongest. The rater must use the test specific rubric and must focus on information related to the first two items as they appear within the student response.`" Z`keHowever, there may be a situation where a student has appropriately applied information about the third item in a discussion on item one or item two. In this case, if the applied information supports the discussion, it should be considered when rating the overall response. $" +'Other Related Questions UWhy do some of the DBQ or Thematic essays use the phrase and/or within the question? :V9" " "  VSED has been using the phrase and/or on social studies essay questions since 1990. The use of and/or benefits students by broadening the scope of acceptable answers and providing a choice of content within the essay. And/or allows the rater to accept the broadest range of student answers as directed by the content-specific scoring rubric. W" Z""." ""t"  {" EXAMPLE: Discuss the political, social, and/or economic causes for the Russian Revolution. By wording the question this way, a teacher can accept a response that discusses any combination of causes. If the task said  and, the student would have to discuss one correct cause from each category. If the task said  or, the student would have to focus on one type of cause only and a rater could not give credit for information on a different type of cause. LZZ"""yHow do raters know how many examples a student should provide when responding to a particular DBQ or Thematic essay task?zzLook for the  S , the plural. This indicates that two or more examples must be provided. However, in the case of a DBQ, a student must also use a minimum number of documents in his/her response and support the essay with as many facts, examples, and details as he/she can. Thus, having more examples usually improves the essay. Collectively, the DBQ documents will always provide multiple examples. @" ZZ jd(If only one cause or impact is asked for, it will usually be phrased  a or  one . Example: a cause of& or one cause of& In general, teachers should teach their students to use as many examples as they can. It adds detail to the essay. It helps show the student s knowledge. " F fbOther Scoring Notes` In Global History, can we rate essay responses in which students wrote about the United States?aaMany topics in Global and U.S. History are intertwined. (The Cold War, The Middle East, Vietnam War, etc.) Not all essays will automatically exclude the use of information on the U.S. as part of an acceptable answer. An example of this can be found on the June 2011 Global History and Geography thematic essay. This can be found online at: http://www.nysedregents.org/GlobalHistoryGeography/611/20110615-gh-rg1.pdf. >" PPTKc0TwpTools required for scoringDirections for Administering Regents Examinations www.p12.nysed.gov/assessment/hsgen H & G Information Booklet for Scoring Regents Examinations in Global History and Geography and United States History and Government www.p12.nysed.gov/assessment/hsgen School Administrator s Manual, Secondary Level Examinations http://www.p12.nysed.gov/assessment/sam/secondary/home.html .8" P>" Pv^c02T_c0c09txqGlobal History and Geography and United States History and Government rating guides for specific test administrations available online (" " yrTest manuals should be reviewed by all administrators, teachers, and staff administering and/or scoring a 鶹ýӳ State examination. Anyone who is involved in the scoring of a 鶹ýӳ State examination must complete the training as specified in the test manuals and rating guides." ZReminder2Teachers are allowed to proctor their own students, unless the district has a different policy. Teachers may not score their own students answer papers." y Training All training should be based on the protocols established by 鶹ýӳ as found in the H & G Information Booklet for Scoring Regents Examinations in Global History and Geography and United States History and Government as well as in the test specific rating guides." zScorers are encouraged to ask questions that relate to understanding the scoring guidelines and questionable papers, or scaffolding responses, but philosophical differences, personal preferences and biases must be set aside during the scoring process." {The goal of scoring is to be sure the scoring criteria in the rubrics are accurately and consistently applied to each student response being rated across 鶹ýӳ State." |-Questions about scoring social studies exams.."Contact the Associates in Social Studies in the Office of State Assessment For Global History and Geography: Donna Merlau, dmerlau@mail.nysed.gov For United States History and Government: Greg Wilsey, gwilsey@mail.nysed.gov Phone: 518-474-0556 rn" '" *" %" " " Pu2c0}c0icYour input is importantFeedback on State examinations is very important to the test development process. This can be done by completing our online Teacher Evaluation Forms at: http://www.forms2.nysed.gov/emsc/osa/exameval/reexameval.cfm B" @" " Ac0r@# &1Root EntrydO)PH@PicturesY%Current User SummaryInformation(      !"#$%&'()*+,-./0123456789:;<=>?@AB0      !"#%*+./t23456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrs$uvwxyz{|}_6 yentnguyen