Summer 2020 NAACCR Narrative Leave a comment

Elizabeth Ward, PhD
Co-Chair, Site Specific Data Item Work Group


Have you ever wished that a standard setter had asked your opinion on codes and coding instructions before releasing a new data item? Or wanted to give feedback on problems in coding an existing data item? Or asked you how available a data item was before implementing a new data item? If yes, now is your opportunity to speak up!


In 2019, the NAACCR Mid-Level Tactical Group (MLTG) mandated that any new data items that were going to be introduced into the registry needed to go through a field study. The purpose of a field study is to test proposed data items, including codes and coding instructions. This was not the first time that field studies had been discussed though. This was a long-term desire by the Cancer Surveillance Community. The problem continued to be how to conduct a field test.


Then Kathy Cronin, the Deputy Associate Director for the Surveillance Research Program at SEER, asked a question that would change everything. While SEER staff were discussing how to implement a field study, Kathy asked about SEER’s Reliability Website. After further discussions, it was agreed by SEER and IMS that it would be possible to modify the Reliability Website to meet the needs of the field study and the proposal to use the SEER Reliability Website was shared with AJCC/CoC, NPCR, NCRA, NAACCR and Canadian Registries. It was agreed upon by all the partners that the SEER Reliability website would be used for the field test in 2019. A small team was put together to gather all the needed information and the process was started. The team was led by Jennifer Ruhl from NCI, and included Nicki Schussler from IMS and Carmela Groves from Westat.


The first step to any field study is getting actual cases from hospitals or central registries. A call for cases goes out with the data items being tested, and the specific information, including sites and/or histology, that are needed. The cases are then received and reviewed by technical experts, in the previous field study that was Jennifer and Carmela. Cases are chosen based on if they contain the needed information, and the difficulty level. Then proposed answers and rationale are developed by the experts and then reconciled. Once Jennifer and Carmela came up with preferred answers, another group of CTR experts were provided by NCRA, who also reviewed the cases, the preferred answers and rationale. During this process, there was also discussions with the SSDI work group, AJCC physicians and pathologists. Finally, the preferred answers and rationale were determined, and the cases went off to IMS, where Nicki did her magic, along with help from some of her IMS colleagues.


For the 2019 Field Study, we tested seven new SSDIs, the new yc data items from AJCC (multiple choice questions only) and SEER’s neoadjuvant data items (multiple choice questions only). Approximately 300 registrars participated in the field test, the majority of them from hospital registries. There were five groups of cases. For the SSDIs, each group had a different medical record, each record with a slightly different way of how the biomarkers (which were being tested) were recorded in the medical record. With this type of approach, it helps those developing the data items get a wider view of how things may be documented and how the instructions should be developed.


For the field test process, registrars entered their answer and immediately were given the “preferred” answer and the rationale for that answer. Registrars were then able to comment on that particular data item or question. Through those comments, we were able to determine where the problem areas were with the data items and make some adjustments to the proposed data items before they were released. Not only did we get comments directly related to the new data items, we also received feedback that is helping the SSDI work group enhance other SSDIs. It is this feedback that is so valuable to us.


It is the goal of the MLTG that at least 500 registrars participate so that we can get more feedback. This year, there will be one proposed new data item and eight existing data items under review. The SSDIs that are already being collected and included in this field study are ones that frequently have questions in the CAnswer Forum, and where clinical changes may be occurring. We are hoping by field testing these, that we may be able to determine where the major issues are and make some adjustments to these SSDIs that will help registrars in the future. For one of the SSDIs, major changes have already been made and we are testing those changes. We are also including cases on Grade. The Grade instructions for the 2021 updates had some major changes and we will be testing those changes as well. (Note: Updated SSDI and Grade manuals will be available in September 2020. Changes to software will not be available until later in 2020).


The call for cases has gone out through a NCRA in a special news flash and also the NAACCR Listserv. We are encouraging all registrars, in particular, hospital registrars to submit cases. These cases are critical to successful field study. Also, having cases from different facilities helps to develop an overall picture of how these data items are documented in different facilities.


The SEER Reliability Website for the Field Study will be open on October 15, 2020, for account creation and registration. Shortly before that, an invitation to participate will also be sent out through NCRA, NAACCR and SEER. The study will be open November 1 -December 15 for participants to complete their cases. And don’t forget, we want and need your feedback (post your comments in the Field Study). One comment can make a huge difference. This happened several times with the 2019 Field Study and we are so appreciative of those registrars.


Participants are asked to complete at least one set of cases. They will have the opportunity to complete all five sets of cases. CE credits will be available for each set completed.


In addition, we will also be having another section this year, which will focus on data items that Cancer Surveillance is considering collecting. We are looking to see if the information is available, and how often it is available. You may be asked to look at the last several cases of a particular primary site to see if certain information is available. This will help Cancer Surveillance determine if something is widely available enough to consider as a new data item.


We know how busy everyone is but really hope that you will take the time to participate in this important effort to improve the process of implementing new data items that are emerging as important for cancer treatment and prognosis.




Copyright © 2016 NAACCR, Inc. All Rights Reserved | See NAACCR Partners and Sponsors