Home Print this page Email this page Users Online: 912
Home About us Editorial board Search Ahead of print Current issue Archives Submit article Instructions Subscribe Contacts Login 


 
 Table of Contents  
ORIGINAL ARTICLE
Year : 2014  |  Volume : 2  |  Issue : 3  |  Page : 190-196

Physicians' override of computerized alerts for contraindicated medications in patients hospitalized with chronic kidney disease


Department of Health Information Management and Technology, College of Applied Medical Sciences, University of Dammam, Dammam, Saudi Arabia

Date of Web Publication11-Oct-2014

Correspondence Address:
Adel Youssef
Department of Health Information Management and Technology, College of Applied Medical Sciences, University of Dammam, Dammam - 31451
Saudi Arabia
Login to access the Email id

DOI: 10.4103/1658-631X.142540

Rights and Permissions
  Abstract 

Objectives: To determine the effectiveness of a clinical decision support system (CDSS) as indicated by a lower proportion of receiving contraindicated medications by patients with severe chronic kidney disease (CKD) compared with patients with less severe CKD.
Materials and Methods: This was a retrospective analysis of inpatients with CKD (ICD9-CM 585.xx) admitted to a major tertiary hospital in Saudi Arabia and receiving one of the medications that were documented in the knowledge base of the hospital CDSS to be renally cleared and/or nephrotoxic. Using the Chi square test, the proportion of receiving contraindicated medication was compared between patients with severe CKD and patients with mild/moderate CKD. Multivariate logistic regression was then used to examine the adjusted risk of receiving contraindicated medications among patients with severe CKD despite the presence of guided medication by CDSS.
Results: The final analysis was conducted on 346 patients who received prescriptions that were renally cleared and/or nephrotoxic. Of these patients, 17% (n = 58) had severe CKD and 83% (n = 288) had mild/moderate CKD. Among patients with severe CKD, 51.7% (n = 30) received contraindicated medications compared with patients with mild/moderate CKD, 4.9% (n = 14), P < 0.01. Multivariate logistic regression showed that the likelihood of receiving contraindicated medications was several folds higher among patients with severe CKD compared with patients with mild/moderate CKD (P < 0.001).
Conclusion: Patients with severe CKD continued to receive contraindicated medications despite the availability of medication guidance by the CDSS to prescribing physicians. Improved compliance by physicians to CDSS alerts and better understanding of reasons for non-compliance is still needed, particularly for patients with severe CKD.

  Abstract in Arabic 

ملخص البحث :

تعنى هذه الدراسة بتحديد فاعلية نظام دعم القرار السريري كما هو مبين عن طريق انخفاض نسبة استخدام أدوية غير مناسبة لمرضى الفشل الكلوي الشديد مقارنة بالمرضى الذين يعانون من ضعف قليل في وظائف الكلى. أظهرت هذه الدراسة الاسترجاعية والتي شملت 643 مريضًا يعانون من قصور في وظائف الكلى ، أن هؤلاء المرضى استمروا في استخدام أدوية غير مناسبة بالرغم من توفر إرشادات للدواء عن طريق (SSDC) للأطباء. ينصح بامتثال الأطباء لتنبيهات (SSDC) خاصة لمرضى الفشل الكلوي الشديد.




Keywords: Computerized decision support system (CDSS), contraindicated medications, override, Saudi Arabia, severe renal insufficiency


How to cite this article:
Alharthi H, Youssef A. Physicians' override of computerized alerts for contraindicated medications in patients hospitalized with chronic kidney disease. Saudi J Med Med Sci 2014;2:190-6

How to cite this URL:
Alharthi H, Youssef A. Physicians' override of computerized alerts for contraindicated medications in patients hospitalized with chronic kidney disease. Saudi J Med Med Sci [serial online] 2014 [cited 2019 Jun 19];2:190-6. Available from: http://www.sjmms.net/text.asp?2014/2/3/190/142540


  Introduction Top


Adverse drug events (ADEs) due to medication error are a worldwide concern to every healthcare provider because of its associated increased morbidity and mortality as well as increased cost of care. [1],[2] While more than half of all preventable medication errors are the consequence of improper physician orders, efforts to reduce medication errors through computerized physician order entry (CPOE) and clinical decision support systems (CDSSs) have the potential to lower the rate of ADEs and improve the overall delivery of care. [3],[4]

CPOE, which allows physicians to directly enter orders into the computer system, eliminates illegible orders, improves communication and improves the tracking of orders. [5] Moreover, when fully integrated with the electronic patient records and with continuous access to up-to-date patient medical information, coupling CPOE with automated CDSS has the capability of reducing medication error by utilizing dosing recommendations, alerting to contraindicated medications and checking for drug - drug and drug - allergy interactions. [6],[7]

CKD, which is increasingly becoming a global public health problem, is relatively common among hospitalized patients. [8] Ferris et al. [9] have reported that 40% of patients admitted to a large academic hospital have some level of CKD. Others have also demonstrated that elderly patients experience worsening renal function during their hospital stay. [10],[11],[12] The high prevalence of CKD and the large number of commonly used drugs with renal elimination or potential nephrotoxicity suggests that physicians should keep track of patient's renal function and be cautious to order medications that are appropriate to the level of kidney impairment. [11],[13]

In one study, one-half of adverse drug reactions related to hospital admissions were due to inappropriate dosing mainly due to practitioners not considering decreased renal function. [14] CDSS have been shown to contribute to error reduction through checking for potential errors and providing alert and recommendations at the time of ordering. [15] However, the effectiveness of CDSS to improve medication prescribing depends on the extent to which physicians acknowledge the alerts presented. Several studies have shown that only small percentages of alerts given by CDSS resulted in modification of the prescription by the physician even in situations when the alert is deemed by the system to be of high severity. [16],[17]

Several studies have examined the general behavior of clinicians regarding medication alerts and factors associated with overriding these alerts in patients with renal impairment. [18],[19],[20] However, these studies may suffer from some bias. Physicians having a sense of security that the CDSS will fire alerts if the medication entered is contraindicated may have the tendency to enter more aggressive orders than they otherwise do in the absence of the alert. The effect of that behavior on the measures of the effectiveness of CDSS is not clear. On the other hand, physicians may choose to override an alert to complete the order and subsequently discontinue the medication before it is delivered to the patient, resulting in an overestimation of the override. [21] Unlike these reports, our study will specifically focus on the effectiveness of the CDSS to reduce the rate of contraindicated medications actually received by the patient irrespective to the number of alerts produced by the system or if the physician eventually accepted the alert. We hypothesized that the incorporation of a guided decision support into the CPOE for medications to patients with CKD should be reflected in a lower administration of contraindicated medications to patients with severe CKD compared with patients with mild/moderate CKD.


  Materials and methods Top


Study site and setting

This study was conducted in a Governmental Hospital in Dammam, Eastern Province, Saudi Arabia. The hospital utilizes commercially available electronic medical records (EMRs) provided by the Ministry of Health and is mandatory to be used by all departments in the hospital. The system integrates the CPOE, updated patient's information from EMR and CDSS allowing physicians to use it as a central source of ordering and reviewing of all lab results. The system in addition provides alerts to support appropriate prescription and patient safety. The system was implemented in 2005; all providers of care are required to use the system. Nurses are not allowed to enter medications through the system.

CDSS and internal logic relevant to CKD

Each time a newly measured serum creatinine is added by the laboratory to the EMR, an estimated glomerular filtration rate (eGFR) is calculated by the system according to the Modification of Diet in Renal Disease formula (MDRD). [22] Utilizing a list of drugs that are either renally cleared and/or nephrotoxic, the internal logic of the CDSS was designed to trigger an alert when the physician attempts to order one of the drugs that are contraindicated according to the specific level of the most recent eGFR and a pre-determined safe cut point for the drug. Alerts could also be generated if the dosage should be adjusted in dose and/or frequency for a patient whose most recent eGFR was less than the safe level of the default dosage of that drug. This commercial package knowledge base utilizes three categories of renal function for its medication adjustment recommendations: eGFR >50 mL/min, eGFR 10-50 mL/min and eGFR <10 mL/min.

The system provides an onscreen alert message with the established recommendations for completing the order in the form of reduced dose or frequency of the medication or to avoid the prescription all together. Physicians can override the alert with no requirement to enter reasons for the override. There is also no hard stop in this system. Alerts are also given during renewal of a medication that was overridden earlier to the patient. Pharmacists can contact the prescribing physician to double-check overridden orders. A screen shot of the alert produced during entering a contraindicated medication is shown in [Figure 1].
Figure 1: Screen shot of an alert that results if the physician enters an order of Aspirin to a patient with chronic kidney disease

Click here to view


Determination of drug administration and alert compliance

Data were extracted from the hospital electronic information system. Patients aged above 18 years and admitted to the inpatient ward of the internal medicine department with a documented diagnosis of CKD (ICD9-CM 585.xx) during 1 year (January 1 st , 2010 through December 31 st , 2010) were eligible to be included in the study. Patient characteristics and all medications received by eligible patients during the study period were retrieved. These data were then mined to determine instances where the treating physician prescribed and completed any of the medications included in the commercial CDSS database that was renally cleared and/or nephrotoxic and actually administered to the patient. These prescriptions were then categorized according to the database logic and patient eGFR at the time of prescription into two categories:

  1. Contraindicated medication, when the physician completed an order of a medication resulting in a patient receiving a contraindicated medication and
  2. Not a contraindicated medication, if the medication prescribed and administered to the patient was not contraindicated.


Patients' renal functions were collapsed into two categories:

  1. eGFR >50 mL/min and eGFR 10-50 mL/min were collapsed into one category "eGFR ≥10 mL/min" and was considered as mild/moderate CKD and
  2. The other category was kept as "eGFR <10 mL/min" and was considered as severe CKD.


Patients <18 years and those who had renal transplant, on dialysis, or diagnosed with acute kidney injury (AKI) were excluded from the study. Patient and physician information was de-identified. The study was approved by the hospital institutional review board.

Statistical analysis

Individual patient was used as the unit of analysis. We first described the study population with mean and standard deviation for continuous variables and proportions for categorical variables. Chi square test was used to compare the characteristics of patients who received at least one contraindicated medication with those who did not receive contraindicated medications. We then fit a logistic regression model to evaluate the hypothesis that under a CDSS environment, patients with severe CKD receive less-contraindicated medication compared with patients with mild/moderate CKD. The dependent variable was whether the patient received a contraindicated drug (Yes vs. No). The independent variables included patient age (categorized for better fit), sex, CKD (severe vs. mild/moderate), day shift of the prescription (8 am to 4 pm vs. 4 pm to 8 am) and day of prescription (weekend vs. weekday). We also included physician type (specialist vs. non-specialist). Because some physicians treated more than one patient and may have prescribed more than one contraindicated drug, robust variance estimates to adjust for having correlated data within physicians was performed. All analyses were conducted using STATA v.11 (StataCorp, College Station, TX, USA).


  Results Top


In this study, a total of 346 patients received at least one prescription that was renally cleared and/or nephrotoxic according to the commercial CDSS knowledge database. As shown in [Table 1], the mean age (SD) of those patients was 53.6 (18.4) years, and 57% were male. Most of the prescriptions included in the analysis were given during the weekdays and in the 8 am-4 pm shift (83% and 63%, respectively), and specialist was the prescriber for 60 patients (17%). Of the study population, 17% (n = 58) had severe CKD and 13% (n = 44) received contraindicated medication at least once. As shown in [Table 2], among patients with severe CKD, 51.7% (30/58) received contraindicated medications compared with patients with mild/moderate CKD, 4.9% (14/288), P < 0.001. Multivariate logistic regression showed that severe CKD was associated with an increased risk of receiving contraindicated medication compared with patients with mild/moderate CKD (OR 21.0, 95% CI 9.54-46.44, P ≤ 0.001). Patients aged >65 years were also at the greater risk of receiving contraindicated prescription compared with younger patients aged <55 years (OR 5.3, 95% CI 2.22-12.68, P < 0.001), [Table 3].
Table 1: Patient characteristics

Click here to view
Table 2: Pattern of completed prescription with contraindicated medications according to patient characteristics and alert type

Click here to view
Table 3: Multivariate logistic regression of factors associated with the risk of patient receiving a contraindicated medication despite system alert

Click here to view



  Discussion Top


In this study, we found that the availability of CPOE with incorporated CDSS in a major hospital did not affect physician's behavior to be more cautious in their orders to patients with severe CKD. As a result, despite the presence of CDSS, patients with severe CKD continued to receive more prescriptions of drugs that should have been avoided compared with patients with mild/moderate CKD.

That association seems counterintuitive, as one would expect that the availability of the CDSS support would help physicians to be more cautious in their medication orders, particularly in patients with severe CKD. Our finding was in contrast to that from Galanter et al.'s [23] study, in which they found that the rate of prescribing contraindicated medications was lower in patients with severe CKD. On the other hand, our study finding was in agreement with that of three other studies conducted in hospitals that interestingly had no CDSS; two of these studies were in an inpatient environment [23],[24] and one study examined drug adjustment to renal function at hospital discharge. [25] Salomon et al., [13] who authored one of these studies, suggested that the unexpected high prescription of contraindicated medications to patients with severe CKD compared with patients with less severe CKD probably does not reflect the quality of the physician order but the fact that the same medication order that is considered as contraindicated in patients with severe CKD is considered as appropriate in patients with mild/moderate CKD.

Other studies on the effectiveness of CDSS alerts in patients with CKD found a range of effectiveness of alerts. Most of these studies acknowledged that CDSS effectiveness were reduced due to physician non-compliance. [17],[19],[26],[27],[28]

Results of physician compliance from our and these studies were also in contrast to that by Shah et al., [29] in which there was a high acceptance of the CDSS alert system by clinicians. The authors in that study suggested that high clinician acceptance of their alerts was achieved by presenting the clinicians with fewer but more meaningful alerts and by categorizing the alerts into severity tiers. [29] Added to the lack of this functionality, the system in our study, not like that by Shah et al., [29] used a commercial knowledge base to derive its alerting. One drawback of using a commercial knowledge base is that it often puts more emphasis on breadth of coverage than on clinical relevance or severity of adverse events. [15],[30] With such a high sensitivity, too many irrelevant alerts may be given leading clinicians to refuse the system altogether due to disruptions in their workflow. [31]

Increased prescription of contraindicated medication to patients with severe CKD in this study may have been because the physician completely disregarded the advice to avoid the order even with patients with severe renal impairment. On the other hand, they may have been reluctant to avoid the drug for patients who critically needed that medication with clinical justification. Outdated or confusing information presented by the system could be another reason for physician non-compliance. In this current system, for example, the cut-off levels of severity are not conforming to the latest KDIGO guidelines. [32] Furthermore, the categories of renal disease severity presented on the screen are without the 1.73 m 2 normalization despite the system use of the MDRD formula to calculate patient's eGFR, which pre-normalizes the calculation. [22] In future studies, it is important to assess perceptions and reasons for override by clinicians who use the system. This information could be utilized in-house to update the knowledge base of the system with a better and more flexible alerting system. [33]

There are a number of limitations in this study. First, we focused on patients with CKD using one commercial system. Thus, it is possible that our results may not be generalizable to other diseases or systems. Second, the setting was inpatient and in a single governmentally affiliated hospital, and results may be different in other settings such as private hospitals, primary care or in outpatient settings. Third, we evaluated alert usefulness based on physician-completed orders of contraindicated drugs in patients with severe CKD, but we did not consider situations where physicians accepted system advice or other actions taken, such as ordering further tests or more close observation of the patient. Fourth, we did not examine clinical relevance of the given drug, which may have been justified on a clinical basis. Finally, this study only examined contraindicated medications that should have been avoided according to the alert system. Actions related to other alerts, such as dose reduction and drug interaction, were not addressed.

In conclusion, our study showed that despite the availability of clinical decision support, patients with severe CKD continued to receive contraindicated medications compared with patients with less severe CKD, indicating that physicians were probably not compliant to CDSS warning alerts in patients with severe CKD. Non-compliance has been a hindrance to achieve maximum benefit from drug alert systems. In the ideal world, systems are required to protect patients from all possible medication dangers, but at the same time they need to be very specific to reduce interruption to physician workflow and reduce alert fatigue. Further research is needed to continue improving CDSS systems and find ways to improve physicians compliance and maximize the full potential benefits of CDSS to reduce medication error and improve overall patient safety.


  Acknowledgments Top


The authors are grateful to Alaa Almubarak for her major effort to facilitate the completion, retrieval of critical parts of the data and assistance in the preparation of the manuscript. The authors would also like to thank Maisan Aljohnai, Malak Alnuaimi, Banan Alshehri, and Ghayda Al-ghamdi for their help in data collection.

 
  References Top

1.
Committee on Quality of Health Care in America, Institute of Medicine. In: Kohn LT, Corrigan JM, Donaldson MS, editors.To Err is Human: Building a Safer Health System, 1 st ed.: Washington, DC, National Academies Press; 2000.  Back to cited text no. 1
    
2.
Bates DW, Spell N, Cullen DJ, Burdick E, Laird N, Petersen LA, et al. The costs of adverse drug events in hospitalized patients. Adverse drug events prevention study group. JAMA 1997;277:307-11.  Back to cited text no. 2
    
3.
Ammenwerth E, Schnell-Inderst P, Machan C, Siebert U. The effect of electronic prescribing on medication errors and adverse drug events: A systematic review. J Am Med Inform Assoc 2008;15:585-600.  Back to cited text no. 3
    
4.
Bobb A, Gleason K, Husch M, Feinglass J, Yarnold PR, Noskin GA. The epidemiology of prescribing errors: The potential impact of computerized prescriber order entry. Arch Intern Med 2004;164:785-92.  Back to cited text no. 4
    
5.
Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review. JAMA 2005;293:1223-38.  Back to cited text no. 5
    
6.
AshJ S, Gorman PN, Seshadri V, Hersh WR. Computerized physician order entry in U.S. hospitals: Results of a 2002 survey. J Am Med Inform Assoc 2004;11:95-9.  Back to cited text no. 6
    
7.
van Wijk MA, van der LeiJ, Mosseveld M, Bohnen AM, van Bemmel JH. Assessment of decision support for blood test ordering in primary care. A randomized trial. Ann Intern Med 2001;134:274-81.  Back to cited text no. 7
    
8.
Verbeeck RK, Musuamba FT. Pharmacokinetics and dosage adjustment in patients with renal dysfunction. Eur J Clin Pharmacol 2009;65:757-73.  Back to cited text no. 8
    
9.
Ferris M, Shoham DA, Pierre-Louis M, Mandhelker L, Detwiler RK, Kshirsagar AV. High prevalence of unlabeled chronic kidney disease among inpatients at a tertiary-care hospital. Am J Med Sci 2009;337:93-7.  Back to cited text no. 9
    
10.
Burkhardt H, Bruckner D, Gladisch R. Risk factors of worsening renal function in hospitalized elderly patients. J Nephrol 2005;18:166-73.  Back to cited text no. 10
    
11.
Fields W, Tedeschi C, Foltz J, Myers T, Heaney K, Bosak K, et al. Reducing preventable medication safety events by recognizing renal risk. Clin Nurse Spec 2008;22:73-80.   Back to cited text no. 11
    
12.
Stevens LA, Viswanathan G, Weiner DE. Chronic kidney disease and end-stage renal disease in the elderly population: Current prevalence, future projections, and clinical significance. Adv Chronic Kidney Dis 2010;17:293-301.  Back to cited text no. 12
    
13.
Salomon L, Deray G, Jaudon MC, Chebassier C, Bossi P, Launay-Vacher V, et al. Medication misuse in hospitalized patients with renal impairment. Int J Qual Health Care 2003;15:331-5.  Back to cited text no. 13
    
14.
McDonnell PJ, Jacobs MR. Hospital admissions resulting from preventable adverse drug reactions. Ann Pharmacother 2002;36:1331-6.  Back to cited text no. 14
    
15.
Lin CP, Payne TH, Nichol WP, Hoey PJ, Anderson CL, Gennari JH. Evaluating clinical decision support systems: Monitoring CPOE order check override rates in the department of veterans affairs'computerized patient record system. J Am Med Inform Assoc 2008;15:620-6.  Back to cited text no. 15
    
16.
Abookire SA, Teich JM, Sandige H, Paterno MD, Martin MT, Kuperman GJ, et al. Improving allergy alerting in a computerized physician order entry system. Proc AMIA Symp2000:2-6.  Back to cited text no. 16
    
17.
Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians' decisions to override computerized drug alerts in primary care. Arch Intern Med 2003;163:2625-31.  Back to cited text no. 17
    
18.
McCoy AB, Waitman LR, Gadd CS, Danciu I, Smith JP, Lewis JB, et al. A computerized provider order entry intervention for medication safety during acute kidney injury: A quality improvement report. Am J Kidney Dis 2010;56:832-41.  Back to cited text no. 18
    
19.
Oppenheim MI, Vidal C, Velasco FT, Boyer AG, Cooper MR, Hayes JG, et al. Impact of a computerized alert during physician order entry on medication dosing in patients with renal impairment. Proc AMIA Symp 2002:577-81.  Back to cited text no. 19
    
20.
Sellier E, Colombet I, Sabatier B, Breton G, Nies J, Zapletal E, et al. Effect of alerts for drug dosage adjustment in inpatients with renal insufficiency. J Am Med Inform Assoc 2009;16:203-10.  Back to cited text no. 20
    
21.
Jani YH, Barber N, Wong IC. Characteristics of clinical decision support alert overrides in an electronic prescribing system at a tertiary care paediatric hospital. Int J Pharm Pract 2011;19:363-6.  Back to cited text no. 21
    
22.
Hojs R, Bevc S, Ekart R, Gorenjak M, Puklavec L. Kidney function estimating equations in patients with chronic kidney disease. Int J Clin Pract 2011;65:458-64.  Back to cited text no. 22
    
23.
Galanter WL, Didomenico RJ, Polikaitis A. A trial of automated decision support alerts for contraindicated medications using computerized physician order entry. J Am Med Inform Assoc 2005;12:269-74.  Back to cited text no. 23
    
24.
Sheen SS, Choi JE, Park RW, Kim EY, Lee YH, Kang UG. Overdose rate of drugs requiring renal dose adjustment: Data analysis of 4 years prescriptions at a tertiary teaching hospital. J Gen Intern Med 2008;23:423-8.  Back to cited text no. 24
    
25.
van Dijk EA, Drabbe NR, Kruijtbosch M, De Smet PA. Drug dosage adjustments according to renal function at hospital discharge. Ann Pharmacother 2006;40:1254-60.  Back to cited text no. 25
    
26.
Chertow GM, Lee J, Kuperman GJ, Burdick E, Horsky J, Seger DL, et al. Guided medication dosing for inpatients with renal insufficiency. JAMA 2001;286:2839-44.  Back to cited text no. 26
    
27.
Galanter WL, Polikaitis A, DiDomenico RJ. A trial of automated safety alerts for inpatient digoxin use with computerized physician order entry. J Am Med Inform Assoc 2004;11:270-7.  Back to cited text no. 27
    
28.
Payne TH, Nichol WP, Hoey P, Savarino J. Characteristics and override rates of order checks in a practitioner order entry system. Proc AMIA Symp 2002:602-6.  Back to cited text no. 28
    
29.
Shah NR, Seger AC, Seger DL, Fiskio JM, Kuperman GJ, Blumenfel dB, et al. Improving acceptance of computerized prescribing alerts in ambulatory care. J Am Med Inform Assoc 2006;13:5-11.  Back to cited text no. 29
    
30.
Reichley RM, Seaton TL, Resetar E, Micek ST, Scott KL, Fraser VJ, et al. Implementing a commercial rule base as a medication order safety net. J Am Med Inform Assoc 2005;12:383-9.  Back to cited text no. 30
    
31.
Ash JS, Berg M, Coiera E. Some unintended consequences of information technology in health care: The nature of patient care information system-related errors. J Am Med Inform Assoc 2004;11:104-12.  Back to cited text no. 31
    
32.
Stevens PE, Levin A. Kidney disease: Improving global outcomes chronic kidney disease guideline development work group members. Evaluation and management of chronic kidney disease: Synopsis of the kidney disease: Improving global outcomes 2012 clinical practice guideline. Ann Intern Med 2013;158:825-30.  Back to cited text no. 32
    
33.
Spina JR, Glassman PA, Belperio P, Cader R, Asch S. Primary care investigative group of the VA Los Angeles Healthcare System. Clinical relevance of automated drug alerts from the perspective of medical providers. Am J Med Qual 2005;20:7-14.  Back to cited text no. 33
    


    Figures

  [Figure 1]
 
 
    Tables

  [Table 1], [Table 2], [Table 3]



 

Top
 
 
  Search
 
Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

 
  In this article
   Abstract
  Introduction
   Materials and me...
  Results
  Discussion
  Acknowledgments
   References
   Article Figures
   Article Tables

 Article Access Statistics
    Viewed1993    
    Printed43    
    Emailed0    
    PDF Downloaded149    
    Comments [Add]    

Recommend this journal


[TAG2]
[TAG3]
[TAG4]