Optimising Diagnosis and Antibiotic Prescribing for Acutely Ill Children in Primary Care (ERNIE2)

Validation of a Vital Signs and Symptoms Decision Tree and the Effect of a Point-of- Care CRP Test, Oxygen Saturation, a Brief Intervention and a Parent Leaflet on Diagnosing, Antibiotic Prescribing Rate and Parental Satisfaction in Acutely Ill Children in Primary Care."

National Institute for Health and Disability Insurance (RIZIV), Belgium, Research Foundation Flanders

Acute illness is the most common presentation of children attending ambulatory care settings. Serious infections (e.g. meningitis, sepsis, pyelonephritis, pneumonia) are rare, but their impact is quite large (increased morbidity, mortality, induced fear in parents and defensive behaviour in clinicians). Early recognition and adequate referral of serious infections are essential to avoid complications (e.g. hearing loss after bacterial meningitis) and their accompanied mortality. Secondly, we aim to reduce the number of investigations, referrals, treatments and hospitalisations in children who are diagnosed with a non-serious infection. Apart from the cost-effectiveness, this could lead to less traumatic experiences for the child and less fear induction for the concerned parent. Finally, we aim to support the clinicians to rationalise their antibiotic prescribing behaviour, resulting in a reduction of antibiotic resistance in the long run.

This study is a combination of a diagnostic and an intervention study. In a previous study, our research team constructed a decision tree, based on signs and symptoms, to identify serious infections in acutely ill children in primary care. This decision tree had an excellent sensitivity of 97% for the detection of serious infections, but would have lead to 12% of all acutely ill children to be referred, which is unacceptably high. Our main research questions are: In addition to measuring clinical signs and symptoms, can new or existing technology help identify serious infections in acutely ill children in ambulatory care? Translated to our study design: "After applying the decision tree, can we further reduce the number of false positives (and thus increase the specificity) and maintain the same level of sensitivity with a C-reactive Protein Point-of-Care test or pulse oximetry? After excluding children with serious illnesses by applying the decision tree, can a Point-of-Care CRP test, a brief intervention together with a parent leaflet, or both, have effect on: A. Primary outcome measure: Serious infection Immediate antibiotic prescribing rates B. Secondary outcome measures: Parental satisfaction Parental concern Use of other diagnostic tests and medical services (including re-consultation) Cost-effectiveness Impact of the communicator style on the effect of the intervention (interaction) Serious infections will be defined as: A hospital admission for more than 24 hours for any of these diagnoses: Sepsis (including bacteraemia) with a pathogenic bacteria isolated from a haemoculture as the reference standard Meningitis (viral or bacterial) with a positive lumbar puncture (pleocytosis in cerebrospinal fluid and identification of a bacteria or a virus) as the reference standard Abscess (with a positive culture) Pneumonia (viral or bacterial) with an infiltrate seen on chest x-ray as the reference standard Osteomyelitis (pathogens from bone aspirate as the reference standard, or if unavailable with a MRI or bone scan suggestive for osteomyelitis) Cellulitis (acute suppurative inflammation of the subcutaneous tissues) Gastro-enteritis with dehydration Complicated urinary tract infection (positive urine culture (>105/ml pathogens of a single species) and systemic effects such as fever) Viral respiratory tract infections complicated by hypoxia (e.g. bronchiolitis) This is a combination of (I) a diagnostic study in ambulatory care (consisting of general practitioners and ambulatory paediatricians) identifying the diagnostic value of signs, symptoms and technological tests using hospital admission more than 24 hours for a serious infection as the main outcome and (II) an open, randomised, controlled trial with randomisation at practice level. Clinicians will be allocated to 1 of these 4 intervention groups (table 1): These interventions will be performed on children testing negative on the decision tree: Use of C-reactive protein (CRP) point of care test and brief intervention and parent leaflet Use of C-reactive protein point of care test Brief intervention and parent leaflet Usual care All clinicians will be trained to properly perform the point-of-care CRP test. When a child tests positive on the 5-stage decision tree, a point-of-care CRP test will be performed in all 4 intervention groups. Only clinicians in group 1 and 3 will be trained in the brief intervention and handling of the parent information leaflet. All clinicians will be asked to measure oxygen saturation by means of a paediatric finger pulse oximeter, which will be provided to all physicians. 2.1 Randomisation To avoid contamination, a clustered randomised study design was chosen. Each cluster is made up of one practice. A minimum condition to participate as a practice is being able to recruit 36 children during 1 year. A first randomisation procedure will create 2 groups: CRP point of care test or no testing. A second randomisation procedure will create 2 more groups: brief intervention and parent information leaflet or no communication-intervention. To ensure a good balance of the different practice structures (solo versus duo/group/community health centre) and practice types (general versus paediatric practices) between the recruitment arms, the randomisation process will be stratified. 2.2 Sample size calculation A. Diagnostic part Based on the binomial distribution, at an expected sensitivity of 97% with a minimal acceptable lower confidence limit of 85%, the minimal number of cases is 59, not taking into account the non-monotonic nature of power as a function of sample size due to discreteness of the binomial distribution.[17] The number of controls, derived through the formula Ncontrols =Ncases [(1-Prev)/Prev],[18] is 7316 at a prevalence of 0.8% and 6497 at a prevalence of 0.9%. A recent developed nomogram to calculate sample size in diagnostic studies was not useful, because prevalences below 0.01 were not computed in the nomogram.[19] B. Therapeutic part In order to detect an absolute reduction in antibiotic prescribing of 15% (from 40% to 25%), with 80% power at a 5% significance level, an individually randomised study will need 656 patients (164 patients per group, 4 groups). If we randomise 72 practices with an intracluster correlation coefficient (icc) of 0.06, this number increases to a total of 1368 patients (19 patients per cluster). In order to allow for a 90% follow-up rate for the primary outcome measure of antibiotic prescribing, and at least 70% well recruiting practices, we need to recruit 2184 patients (104 practices, 21 per practice and 546 patients in each cell). 2.3 Practices We intend to recruit 92 general practices and 12 paediatric practices. In every group practice, at least 2 practitioners will be recruiting. The recruited practices will be situated throughout Flanders, in urban and rural areas. Physicians will be excluded if the assumption of consecutive inclusion was violated (inclusion of less than two children in 1 month time). Physicians will be asked to complete a survey on communication skills to stratify the intervention groups, according to communicator style. After formal approval, physicians will be asked to provide us with their latest report on antibiotic prescribing profile, as provided by the Belgian National Health and Disability Insurance (RIZIV). 2.4 Patients Patients aged 1 month to 16 years with an acute illness for a maximum of 5 days are included consecutively. Children are excluded if the acute episode was caused by a merely traumatic or neurological illness, intoxication, psychiatric or behavioural problem without somatic cause, or an exacerbation of a known chronic condition. If a physician includes children twice in the study within 5 days, the second registration is considered a repeated measurement on the same subject and is subsequently excluded from the analysis. 2.5 CRP point of care test A pilot study was performed to determine the user-friendliness, technical accuracy, inter-device and inter-observer variability of the selected point-of-care CRP devices. Based on a recent horizon scanning report by the Monitoring and Diagnosis Group in Oxford (Department of Primary Health Care) and thorough review of the existing technologies in point-of-care CRP devices through industry contacts, we assembled an up-to-date list of all available devices. For use in children the sample size had to be less than 5 µl and the test duration less than 5 minutes. Two point-of-care CRP devices (Afinion™ AS100 Analyzer by Axis-Shield and the LifeAssays™ Reader by LifeAssays) met our criteria and were included in the pilot study. We evaluated both devices based on these criteria: device start-up, test duration, sample size, test characteristics, calibration, measuring range, handling of test tubes, filling of the capillary, placing the test cartridge, test recording and additional material usage. CRP values were measured on two different point-of-care CRP devices in 100 children and compared with the laboratory CRP results in hospitalised children and children consulting an outpatient paediatric clinic of a university hospital. To verify whether the devices were user-friendly and reliable, a CRP measurement was performed on children and adults attending in 5 General Practice surgeries by 2 different Point-of-Care CRP devices. The inter-device (by 1 physician) and inter-observer (by two physicians) variability was tested on 10 patients in one GP surgery each on two different POC devices of the same brand. During the pilot study, one device failed to meet these standards of user-friendliness and test accuracy and was deemed unsuitable for further use in our study. (LifeAssays™ Reader) A perfect correlation was found between the results from the other point-of-care CRP device (Afinion™ Analyzer) and the laboratory results. No significant inter-observer and inter-device variability was found. The device was well received by the users and scored high on user-friendliness. All clinicians will have a CRP test device (Afinion™, Axis-Shield) at their disposal. CRP analysis will be performed during the consultation in accordance with the manufacturer's instructions. The Afinion™ CRP Test Cartridge consist of a 1.5 µL glass capillary to be filled with blood from a finger prick and a reagent container. First, the sample is automatically diluted with a liquid that also lyses the blood cells. If the sample is whole blood, the haematocrit value is estimated from transmission measurement of the haemoglobin. In this case, the measured CRP value will be automatically corrected according to the sample's haematocrit (Hct) level, and the result is displayed as the serum-CRP concentration. Then, the sample mixture is aspirated through the membrane coated with anti-CRP antibodies, and all CRP in the sample is concentrated onto this membrane. The conjugate solution containing anti-CRP antibodies labelled with ultra-small gold particles is then sucked through the membrane. The gold-antibody conjugate binds to the immobilized CRP on the membrane, which will turn red-brown. Excess gold-antibody conjugate is removed by a washing solution. The analyser measures the colour intensity of the membrane, which is proportional to the amount of CRP in the sample. The CRP concentration is displayed on the screen. The result is available within 4 minutes. The CRP measuring range is 8-200 mg/L. The results are displayed in intervals of 1 mg/L. If the CRP concentration in a whole blood sample exceeds the range 8-200 mg/L, CRP >200 mg/L will be displayed on the screen. The haematocrit measuring range is 20-60%. If the Hct value is outside this range, an information code will be displayed on the screen. For internal quality a control positive will be measured to confirm the efficacy and correct performance of the test in the beginning and the middle of the study period. The CRP point of care test will be performed in case of a positive decision tree (irrespective of the intervention group) and also in case of a negative decision tree by clinicians of intervention group 1 and 2. Because no reliable cut-off points for CRP are known currently (as this is the aim of this study) for acute infections in children in primary care (nor for referral, nor for prescription of antibiotics), clinicians will not be given guidance on the interpretation of the CRP results. We will not impose restrictions on the clinicians about treatment, other technical investigations nor referrals. The device distributor will provide technical assistance. All clinicians will be trained in the use of the CRP device prior to the start of the study. 2.6 Finger Pulse Oximeter All physicians will be asked to perform a measurement of the oxygen saturation on all children and enter the results on the case report form. The selected device is a clip-on system suitable for use in children, which measures oxygen saturation in the capillary blood as well as the pulse rate. 2.7 Brief intervention and parent information leaflet Brief interventions are commonly used to give opportunistic advice, discussion, negotiation or encouragement. Mostly they take between 5 to 10 minutes. The effect of brief interventions is studied most extensively in the area of smoking cessation, alcohol misuse and increasing physical activity. We will use the brief intervention technique to serve another purpose, namely not to change the patient's, but rather the clinician's behaviour. Our goal is to reduce the numbers of antibiotic prescriptions. As described before, parents have a lot of concerns about the illness of their child and mostly they do not succeed in communicating these concerns clearly to their physician. Our hypothesis is that clinicians sometimes feel as, although they try to reassure the parents that the child will get better with symptomatic treatment, they do not convince them enough and prescribe antibiotics to reach adequate reassurance whenever the slightest doubt exists that the child benefits from the medication. To meet this gap, we developed a brief intervention: we'll ask the clinicians to ask 3 specific questions, namely "Are you concerned?", "What exactly concerns you?" and "Why does this concern you?". This intervention is easy to implement in daily practice and no additional training is required. Secondly, we aim to reduce the improper demand for antibiotics by the parents. As described before, parents contact their clinician when they feel they are losing control over the illness, e.g. when the child is having very high fever they cannot get down, or the child doesn't stop crying, or doesn't want to eat or sleep anymore. Mostly they already tried some methods to reduce fever or other symptoms, but failed in taking control over the situation. They still believe that antibiotics are the strongest medication you can get and hope to get them from the clinician when they visit him. We developed a parent information leaflet that gives information about what they can do when their child is ill, which signs are important to follow up and when they really should get advice from a clinician. This could make it easier for them to cope with an ill child. The clinician could use this leaflet to give advice and make clear when they have to re-consult their physician to re-evaluate the child. Our hypothesis is that through creating this safety net, the improper demand for antibiotics could be reduced. 2.8 Data collection Acutely ill children (0 to 16 years old) presenting in ambulatory care (general practitioner or paediatrician) will be asked to participate in this trial. Eligible children (if applicable) and their parents will receive an information leaflet. The clinician will ask them to sign the informed consent form. Every participating child is allocated to an individual research number. Only the number, and not the name of the child, will be mentioned on the registration forms to ensure anonymity. Only the clinician and research assistant will know which child correlates to which number. This is needed to ensure the follow up of the child. The name of the child will not be mentioned in a database or scientific publication. During history taking, clinicians participating in group 1 and group 3 will ask the brief intervention questions. After a thorough clinical examination, including measurement of the oxygen saturation, the clinician runs the decision tree. If the child scores positive, he will execute a POC-CRP-test. If the child scores negative, the intervention will depend on the intervention arm: clinicians participating in group 1 and 2 will execute a POC-CRP-test anyway, clinicians participating in group 1 and 3 will hand over the parent information leaflet and clinicians participating in group 4 will pursue usual care. The clinician records the gathered data, working diagnosis and planned actions (e.g. investigations, treatment) on a registration form. After the consultation, parents complete a survey about parental concern and satisfaction about the consultation process and the clinician's communication, which will be posted in a sealed box at the clinician's office. This survey includes the 26 questions of the "Parental Medical Interview Satisfaction Scale"(7 points Likert scale)[20], 4 questions about communication of the "Parent's perceptions of Primary Care items" (5 points Likert scale).[21] Both instruments were translated into Dutch, which was ensured through a forward-back-translation process. After translation, they were validated again in a pilot study. The parental concern survey was developed by our research team and consists of 2 Visual Analog Scales about concern before and after the consultation and a short questionnaire about the content of the concern. This survey was also validated in a pilot study. At the end of the consultation, parents will get a follow up diary, which should be filled in daily, namely a 4 item scale about degree of illness (evaluated by the parent) and parental concern, fever and eating, drinking, crying, playing and sleeping behaviour. Additionally, we will ask the parents to complete questions about educational level, social background and whether they consulted another physician during the same illness episode. If parents score all items of the scale as "no problem", the child is considered as "cured". At that time, we'll ask the parents to send us a short text message. Parents who do so, will participate in a monthly lottery (motivational gift). Parents will be able to forward the diary to the clinical research associate or bring them to their GP in a prepaid envelope. Participating clinicians will complete a questionnaire about their defensive attitude once during the study period. 2.9 Patient follow up Our research team will register the children as "cured" after receiving the short text message or the diary. If needed, reminders to complete the study documents will be sent by post after 14 days. If there is no response, the research team will contact the parents by telephone (at least 3 attempts). In the last case, the clinician, who initially recorded the case, will be contacted to verify if the child completely healed or was referred to the hospital. In case of referral and/or hospitalisation, the primary care and hospital clinicians will be contacted to reconstruct the illness episode (reason for referral/hospitalisation, diagnosis, time to cure). 3.0 Analysis Primary outcome measure: Immediate antibiotic prescribing rates This will be registered by the clinician on the CRF and a Chi square or Fisher's exact test will determine the significance of the difference in results between the intervention groups. The results will be adjusted for relevant patient- or clinician-based characteristics using multiple logistic regression. Diagnostic Accuracy of 5-stage decision tree and added value of technology (point-of-care CRP testing and oxygen saturation) The diagnostic accuracy of the 5-stage decision tree will be tested and reported in sensitivity, specificity, positive and negative likelihood ratios, and positive and negative predictive values. Whenever possible, Receiver Operation Characteristic (ROC) curves will be plotted. The value of the point-of-care CRP test and the oxygen saturation will each be added to the 5-stage decision tree and compared to the results of the 5-stage decision tree alone, to determine the added value of the added technology in terms of sensitivity, specificity, positive and negative likelihood ratios, and positive and negative predictive values. Secondary outcome measures: Parental satisfaction This will be registered by the parents through a questionnaire (see above). The results will be compared between the intervention groups, evaluating their significance through a Chi square or Fisher's exact test. Parental concern This will be registered by the parents through a parental concern satisfaction survey (see above). The results will be compared between the intervention groups, evaluating their significance through a Chi square or Fisher's exact test. Use of other diagnostic tests and medical services (including re-consultation) This will be registered by the clinician on the CRF and checked by the research team. (including the number of re-consultations) The time to complete cure will be evaluated through registration by the parents in the fever diary and checked by the research team. The results will be compared between the intervention groups, evaluating their significance through a Chi square or Fisher's exact test. Cost-effectiveness A preliminary cost-effectiveness study will be performed evaluating the costs related to use of diagnostic tests and medical services and hospital costs compared to the cost of applying the requested interventions and the possible effect on antibiotic prescribing rates. Influence of the communicator style on the effect of the intervention (interaction) This will be registered by the clinician on a communication survey. The results will be stratified according to the communicator styles and compared between the intervention groups, evaluating their significance through a Chi square or Fisher's exact test.

Sepsis, Bacteraemia, Meningitis, Abscess, Pneumonia, Osteomyelitis, Cellulitis, Gastro-enteritis With Dehydration, Complicated Urinary Tract Infection, Viral Respiratory Infection Complicated With Hypoxia

CRP analysis will be performed during the consultation in accordance with the manufacturer's instructions. The CRP point of care test will be performed in case of a positive decision tree (irrespective of the intervention group) and also in case of a negative decision tree by clinicians of intervention group 1 and 2. Because no reliable cut-off points for CRP are known currently (as this is the aim of this study) for acute infections in children in primary care (nor for referral, nor for prescription of antibiotics), clinicians will not be given guidance on the interpretation of the CRP results. We will not impose restrictions on the clinicians about treatment, other technical investigations nor referrals. The device distributor will provide technical assistance. All clinicians will be trained in the use of the CRP device prior to the start of the study.

Combination of CRP POC test and the brief intervention & parent information leaflet intervention groups (factorial design)

Brief interventions are commonly used to give opportunistic advice, discussion, negotiation or encouragement. Mostly they take between 5 to 10 minutes. We developed a brief intervention: we'll ask the clinicians to ask 3 specific questions, namely "Are you concerned?", "What exactly concerns you?" and "Why does this concern you?". This intervention is easy to implement in daily practice and no additional training is required. We developed a parent information leaflet that gives information about what they can do when their child is ill, which signs are important to follow up and when they really should get advice from a clinician. This could make it easier for them to cope with an ill child. The clinician could use this leaflet to give advice and make clear when they have to re-consult their physician to re-evaluate the child. Our hypothesis is that through creating this safety net, the improper demand for antibiotics could be reduced.

All physicians will be asked to perform a measurement of the oxygen saturation on all children and enter the results on the case report form. The selected device is a clip-on system suitable for use in children, which measures oxygen saturation in the capillary blood as well as the pulse rate.

To verify if the child had a serious infection after consulting the primary care physician, the research team will consult all hospital records within the referral region of the physicians to check whether the child was admitted to hospital with a serious infections, based on the appropriate reference standards. In case of referral and/or hospitalisation, the primary care and hospital clinicians will be contacted to reconstruct the illness episode (reason for referral/hospitalisation, diagnosis, time to cure).

The recruiting physicians are asked to fill out whether and which kind of antibiotics they prescribed during the assessment of the sick child. The registration forms will be assessed at 1 year, when the study recruitment phase ends.

Inclusion Criteria: Patients aged 1 month to 16 years with an acute illness for a maximum of 5 days are included consecutively. Exclusion Criteria: Children are excluded if the acute episode was caused by a merely traumatic or neurological illness, intoxication, psychiatric or behavioural problem without somatic cause, or an exacerbation of a known chronic condition. If a physician includes children twice in the study within 5 days, the second registration is considered a repeated measurement on the same subject and is subsequently excluded from the analysis.

Van den Bruel A, Aertgeerts B, Bruyninckx R, Aerts M, Buntinx F. Signs and symptoms for diagnosis of serious infections in children: a prospective study in primary care. Br J Gen Pract. 2007 Jul;57(540):538-46.

Bruyninckx R, Aertgeerts B, Bruyninckx P, Buntinx F. Signs and symptoms in diagnosing acute myocardial infarction and acute coronary syndrome: a diagnostic meta-analysis. Br J Gen Pract. 2008 Feb;58(547):105-11. doi: 10.3399/bjgp08X277014.

Thompson M, Van den Bruel A, Verbakel J, Lakhanpaul M, Haj-Hassan T, Stevens R, Moll H, Buntinx F, Berger M, Aertgeerts B, Oostenbrink R, Mant D. Systematic review and validation of prediction rules for identifying children with serious infections in emergency departments and urgent-access primary care. Health Technol Assess. 2012;16(15):1-100. doi: 10.3310/hta16150.

Van den Bruel A, Haj-Hassan T, Thompson M, Buntinx F, Mant D; European Research Network on Recognising Serious Infection investigators. Diagnostic value of clinical features at presentation to identify serious infection in children in developed countries: a systematic review. Lancet. 2010 Mar 6;375(9717):834-45. doi: 10.1016/S0140-6736(09)62000-6. Epub 2010 Feb 2.

Van den Bruel A, Thompson MJ, Haj-Hassan T, Stevens R, Moll H, Lakhanpaul M, Mant D. Diagnostic value of laboratory tests in identifying serious infections in febrile children: systematic review. BMJ. 2011 Jun 8;342:d3082. doi: 10.1136/bmj.d3082.

Van den Bruel A, Bruyninckx R, Vermeire E, Aerssens P, Aertgeerts B, Buntinx F. Signs and symptoms in children with a serious infection: a qualitative study. BMC Fam Pract. 2005 Aug 26;6:36. doi: 10.1186/1471-2296-6-36.

Buntinx F, Mant D, Van den Bruel A, Donner-Banzhof N, Dinant GJ. Dealing with low-incidence serious diseases in general practice. Br J Gen Pract. 2011 Jan;61(582):43-6. doi: 10.3399/bjgp11X548974.

Bjerrum L, Gahrn-Hansen B, Munck AP. C-reactive protein measurement in general practice may lead to lower antibiotic prescribing for sinusitis. Br J Gen Pract. 2004 Sep;54(506):659-62.

Flood RG, Badik J, Aronoff SC. The utility of serum C-reactive protein in differentiating bacterial from nonbacterial pneumonia in children: a meta-analysis of 1230 children. Pediatr Infect Dis J. 2008 Feb;27(2):95-9. doi: 10.1097/INF.0b013e318157aced.

Cals JW, Chappin FH, Hopstaken RM, van Leeuwen ME, Hood K, Butler CC, Dinant GJ. C-reactive protein point-of-care testing for lower respiratory tract infections: a qualitative evaluation of experiences by GPs. Fam Pract. 2010 Apr;27(2):212-8. doi: 10.1093/fampra/cmp088. Epub 2009 Dec 18.

Francis NA, Butler CC, Hood K, Simpson S, Wood F, Nuttall J. Effect of using an interactive booklet about childhood respiratory tract infections in primary care consultations on reconsulting and antibiotic prescribing: a cluster randomised controlled trial. BMJ. 2009 Jul 29;339:b2885. doi: 10.1136/bmj.b2885.

Kerr J, Swann IJ, Pentland B. A survey of information given to head-injured patients on direct discharge from emergency departments in Scotland. Emerg Med J. 2007 May;24(5):330-2. doi: 10.1136/emj.2006.044230.

Parsley J, Fletcher L, Mabrook AF. Head injury instructions: a time to unify. J Accid Emerg Med. 1997 Jul;14(4):238-9. doi: 10.1136/emj.14.4.238.

Zemek RL, Bhogal SK, Ducharme FM. Systematic review of randomized controlled trials examining written action plans in children: what is the plan? Arch Pediatr Adolesc Med. 2008 Feb;162(2):157-63. doi: 10.1001/archpediatrics.2007.34.

Almond S, Mant D, Thompson M. Diagnostic safety-netting. Br J Gen Pract. 2009 Nov;59(568):872-4; discussion 874. doi: 10.3399/bjgp09X472971. No abstract available.

Chu H, Cole SR. Sample size calculation using exact methods in diagnostic test studies. J Clin Epidemiol. 2007 Nov;60(11):1201-2; author reply 1202. doi: 10.1016/j.jclinepi.2006.09.015. Epub 2007 Aug 3. No abstract available.

Flahault A, Cadilhac M, Thomas G. Sample size calculation should be performed for design accuracy in diagnostic test studies. J Clin Epidemiol. 2005 Aug;58(8):859-62. doi: 10.1016/j.jclinepi.2004.12.009.

Carley S, Dosman S, Jones SR, Harrison M. Simple nomograms to calculate sample size in diagnostic studies. Emerg Med J. 2005 Mar;22(3):180-1. doi: 10.1136/emj.2003.011148. Erratum In: Emerg Med J. 2005 May;22(5):392.

Lewis CC, Scott DE, Pantell RH, Wolf MH. Parent satisfaction with children's medical care. Development, field test, and validation of a questionnaire. Med Care. 1986 Mar;24(3):209-15. doi: 10.1097/00005650-198603000-00003.

Seid M, Stevens GD, Varni JW. Parents' perceptions of pediatric primary care quality: effects of race/ethnicity, language, and access. Health Serv Res. 2003 Aug;38(4):1009-31. doi: 10.1111/1475-6773.00160.

De Rop L, De Burghgraeve T, De Sutter A, Buntinx F, Verbakel JY. Point-of-care C-reactive protein test results in acute infections in children in primary care: an observational study. BMC Pediatr. 2022 Nov 4;22(1):633. doi: 10.1186/s12887-022-03677-5.

Verbakel JY, Lemiengre MB, De Burghgraeve T, De Sutter A, Aertgeerts B, Bullens DMA, Shinkins B, Van den Bruel A, Buntinx F. Point-of-care C reactive protein to identify serious infection in acutely ill children presenting to hospital: prospective cohort study. Arch Dis Child. 2018 May;103(5):420-426. doi: 10.1136/archdischild-2016-312384. Epub 2017 Dec 21.

Verbakel JY, Lemiengre MB, De Burghgraeve T, De Sutter A, Aertgeerts B, Shinkins B, Perera R, Mant D, Van den Bruel A, Buntinx F. Should all acutely ill children in primary care be tested with point-of-care CRP: a cluster randomised trial. BMC Med. 2016 Oct 6;14(1):131. doi: 10.1186/s12916-016-0679-2. Erratum In: BMC Med. 2017 May 2;15(1):93.

Verbakel JY, Lemiengre MB, De Burghgraeve T, De Sutter A, Aertgeerts B, Bullens DM, Shinkins B, Van den Bruel A, Buntinx F. Validating a decision tree for serious infection: diagnostic accuracy in acutely ill children in ambulatory care. BMJ Open. 2015 Aug 7;5(8):e008657. doi: 10.1136/bmjopen-2015-008657.

Lemiengre MB, Verbakel JY, De Burghgraeve T, Aertgeerts B, De Baets F, Buntinx F, De Sutter A; ERNIE 2 collaboration. Optimizing antibiotic prescribing for acutely ill children in primary care (ERNIE2 study protocol, part B): a cluster randomized, factorial controlled trial evaluating the effect of a point-of-care C-reactive protein test and a brief intervention combined with written safety net advice. BMC Pediatr. 2014 Oct 2;14:246. doi: 10.1186/1471-2431-14-246.

Verbakel JY, Lemiengre MB, De Burghgraeve T, De Sutter A, Bullens DM, Aertgeerts B, Buntinx F; ERNIE 2 collaboration. Diagnosing serious infections in acutely ill children in ambulatory care (ERNIE 2 study protocol, part A): diagnostic accuracy of a clinical decision tree and added value of a point-of-care C-reactive protein test and oxygen saturation. BMC Pediatr. 2014 Oct 2;14:207. doi: 10.1186/1471-2431-14-207.