Evaluating the strength or persuasiveness of epidemiologic evidence is inherently challenging, both for those new to the field and for experienced researchers. There are a myriad of potential biases to consider, but little guidance about how to asses the likely impact on study results. This book offers a strategy for assessing epidemiologic research findings, explicitly describing the goals and products of epidemiologic research in order to better evaluate it successes and limitations. The focus throughout is on practical tools for making optimal use of available data to assess whether hypothesized biases are operative and to anticipate concerns at the point of study design in order to ensure that needed information is generated. Specific tools for assessing the presence and impact of selection bias in both cohort and case-control studies, bias from non-response, confounding, exposure measurement error, disease measurement error, and random error are identified and evaluated. The potential value of each approach as well as its limitations are discussed, using examples from the published literature. Such information should help those who generate and interpret epidemiologic research to apply methodological principles more effectively to substantive issues, leading to a more accurate appraisal of the current evidence and greater clarity about research needs. More detailsInterpreting Epidemiologic Evidence: Strategies for Study Design and Analysis By David A. Savitz Published by Oxford University Press US, 2003 ISBN 019510840X, 9780195108408 321 pages |
Wednesday, November 12, 2008
Free Preview Interpreting Epidemiologic Evidence: Strategies for Study Design and Analysis
Interpreting Epidemiologic Evidence: Strategies for Study Design and Analysis (Hardcover)
Interpreting Epidemiologic Evidence: Strategies for Study Design and Analysis (Hardcover)
by David A. Savitz (Author) "This book was written for both producers and consumers of epidemiologic research, though a basic understanding of epidemiologic principles will be necessary at the outset..." (more)
Key Phrases: New York, Oxford University Press, United States (more...)
Interpreting Epidemiologic Evidence: Strategies for Study Design and Analysis. David A Savitz.
Interpreting Epidemiologic Evidence: Strategies for Study Design and Analysis. David A Savitz. Oxford: Oxford University Press, 2003, pp. 336, £34.50 (HB). ISBN: 0-19-510840-X.
Isobel Bray
My first impressions of this book were that while the topic seemed important, the text was perhaps too wordy to be easily accessible. By the end, however, I was convinced that this is a book every epidemiologist should read, or at least use as a reference text when designing and analysing studies. My initial judgement that it would be heavy-going for a new student of epidemiology remains unchanged, but the author states from the outset that this was not intended to be yet another standard text on epidemiology. Rather, the aim is to offer a comprehensive strategy for assessing the results of epidemiological studies.
The book focuses on the potential sources of bias in any study. For example, what are the most likely magnitude and direction of bias due to non-response, confounding, measurement error in the exposure variable, and misclassification of the outcome of interest? This text provides the knowledge and tools necessary to answer these questions, and many more, so that a researcher can make the most of limited resources when planning a study and learn as much as possible from existing data. The author draws heavily on the published literature (particularly in the fields of environmental and occupational epidemiology) to illustrate each point, which makes the text much easier to understand and digest. There are also plenty of methodological references for those wishing to study the issues raised in more detail.
The 12 chapters are quite long, but they are broken down into clearly labelled sections listed in the Contents. This, and the detailed index, should ensure that researchers find the sections of most interest to them quite quickly. Also, each of the central chapters (4–11) ends with a summary of the proposed strategies, referred to as an integrated assessment. To those who have the time, however, I would recommend reading the book cover to cover as the chapters form a logical sequence and cover a range of important issues barely touched on in most taught courses in epidemiology.
The Introduction provides a clear justification for the book and introduces the major themes of bias and confounding. The key message is that just because we cannot quantify these potential problems in our study, we should not fail to attempt to assess their impact. Chapter 2 (The Nature of Epidemiologic Evidence) begins by elucidating the differences between public health and epidemiology. Though this chapter is not a clear introduc-tion to epidemiology for a novice, it makes many important points (e.g. evidence of no association between two variables is equally valid as evidence for a positive association). Chapter 3 (Strategies for Drawing Inferences from Epidemiologic Evidence) sets out the goal of making evaluation of epidemiological evidence more objective. To this end, we should concentrate on the most likely sources of error rather than those that are easiest to measure. There follow substantial chapters on each of the following major themes—selection bias in both cohort and case-control studies, bias due to loss of study participants, confounding, measurement and classification of both exposure and disease, and random error. Chapter 11 (Integration of Evidence Across Studies) introduces the concepts of comparative analysis, meta-regression and narrative review. The final chapter discusses issues such as how epidemiologists should present their results to those who are interested (e.g. policy makers) but may not have the time or inclination to consider the complex assessments of bias described in this book, and how we should interpret conflicting evidence. It also revisits Hill's criteria for causality,1 setting them in a wider context which involves scrutinizing possible sources of error for all studies conducted, not just those which provide evidence of an association.
In conclusion, this book raises many important points that, currently, are largely overlooked in the interpretation of epidemiological data. In this age of data overload, Savitz encoura-ges us to scrutinize the quality of these data in as objective way as possible. I will certainly be referring to this text in the future, and would recommend it as highly relevant to any practising epidemiologist.
Reference
1 Hill AB. The environment and disease: association or causation? Proc R Soc Med 1965;58:295–300.[ISI][Medline]
Thursday, October 30, 2008
GUIDELINES ON CONFIDENTIALITY
This booklet sets out the MMC’s guidelines on confidentiality. It is an extension of the principles described in the booklet `Good Medical Practice’.
read more: http://www.permai.gov.my/panduan/GUIDELINES%20ON%20CONFIDENTIALITY.pdf
Patient Confidentiality Is Priority With Electronic Practice Records
The Medical Defence Union (MDU), the UK's leading medical defence organisation, says that GP practices are regularly seeking its advice on how to ensure patient confidentiality when using electronic records. In an article published in its latest In Practice1 journal for practice managers, MDU medico-legal adviser, Dr James Armstrong, answers some of the most common questions on the subject to the MDU's medico-legal advice helpline in recent years.
Dr Armstrong said: "Electronic records and communications are an increasing feature of GP practice and members may justifiably feel they are a valuable and efficient tool. However, compared to paper records, they do pose slightly different challenges when it comes to safeguarding patient confidentiality and MDU members regularly contact us for risk management advice about this."
Areas addressed by Dr Armstrong include emailing correspondence about patients; protecting information when transferring records electronically; upgrading computer systems; and the need to scan hospital correspondence. One example and the key points of his advice is set out below:
Can we send correspondence about patients by email?
- where an email contains patient information, you must consider the risk of the email being intercepted and ensure that it is effectively protected against improper disclosure at all times.
- any email exchange with or about a patient should take place only with the patient's full agreement, in line with the Data Protection Act 1998.
- you have a responsibility to ensure robust security measures are in place, such as encryption of data.
- it is prudent to satisfy yourself that adequate security is in place at the receiving end.
- The GMC says2 that if necessary you should seek appropriate authoritative professional advice on how to keep information secure and you should record the fact you have taken such advice.
Elsewhere in this issue, MDU medico-legal adviser, Sue Parker, explains how to set up protocols in general practice to define roles and responsibilities within the healthcare team. She said: "As nurses' roles have extended to include more complex medical services and healthcare assistants have become more involved in the delivery of care, protocols can clarify what is expected of those involved and define when they should seek advice from a doctor."
Employees Can be Liable for Violating Patient Confidentiality
Clifford’s Notes, Chicago Lawyer, 05/01/2006
By Robert A. Clifford
Suzanne Bagent, a patient at Quincy Medical Group in September 2003, had blood samples drawn, which were sent to Illini Hospital. Misty Young, a phlebotomist and employee of the hospital, examined the results and later, at a tavern, told Bagent’s sister that the patient was pregnant.
Bagent filed a lawsuit against Illini Hospital and Young, alleging breach of health care practitioner/patient confidentiality, invasion of privacy, and negligent infliction of emotional distress, as well as intentional infliction of emotional distress as to Young.
The plaintiff’s action was grounded in statute (735 ILCS 5/8-802, 2004; 210 ILCS 85/6.17(b) (West, 2004) (patient’s medical information must be protected from disclosure), as well as common law right-to-privacy theories.
Admitting that she revealed the test results in a private conversation, Young filed a motion for summary judgment and accepted the hospitals’ offer of resignation in lieu of termination. Illini Hospital also moved for summary judgment, arguing that Young’s statements were not attributable to the hospital because her actions were not within the scope of her employment and because Young had been trained in confidentiality.
The trial court granted both of the defendants’ motions for summary judgment, but granted plaintiff’s motions for summary judgment against Young on the issue of whether she had improperly revealed confidential information. The issues of intentional infliction of emotional distress and damages also were to be determined at trial. On appeal, the rulings in favor of the defendants were reversed.
The appellate court found that Young had a continuing duty to maintain the confidentiality of patient records. "An employee entrusted with confidential information in the course of his or her employment has a duty not to disclose the information – without limitation as to time or space." Bagent v. Illini Community Hospital, and Misty Young, No. 4-05-0495 (4th Dist., decided March 3. 1006).
As for whether the hospital was liable under the doctrine of respondeat superior, the court said: "Hospital employees, as well as employees of lawyers, therapists, and other employers who maintain confidential information, have a constant duty to keep confidentiality." Therefore, the court held, it is up to a jury to decide whether Young was acting within the scope of employment, on the basis that her disclosure was actuated in part by a purpose to serve the employer. Id.
Although the plaintiff did not specifically plead the federal patient privacy statute, the Health Insurance Portability and Accountability Act of 1996 (HIPPA) (Pub. L. No. 104-191, 110 Stat. 1936), the act would expose the hospital to liability for fines by the Department of Health and Human Services for the actions of its employees.
Patients have a right to insist on privacy. It is their lives that become open books to medical professionals who count on openness and honesty in order to make proper diagnoses.
Take, for instance, the recently filed case of a Palm Beach, Fla., woman who sued Walgreens for defamation. Fifty-three-year-old Janey Karp has battled depression for years with the help of prescription drugs. She felt the need to be private about her problem because of the stigma she sensed in the general for those who needed medical to be "normal."
Much to her shock, when she picked up her prescription about a year ago, she noticed that in the patient information field someone had typed "CrAzY!" In looking at a previous prescription, she read that someone had typed, "She’s really a psycho!! Do not say her name too loud, never mention her meds by names & try to talk to her when..." The information continued on another page that was not attached.
When Karp read this, she knew her worst fears had been realized. She contacted a lawyer who said she made dozens of calls to Walgreens but received a "generic" response. Given that pharmacists and pharmacy technicians across the country have access to these records, Karp said she does not know how many people could have read these notes.
She said she was humiliated, embarrassed, and devastated, so she sued the Deerfield-based company in March, alleging defamation, negligent supervision, and intentional inflection of emotional distress. At the time of the filing, a Walgreens representative stated, "We want to ensure that our pharmacy employees are acting in a proper and professional manner, so we are looking into this matter."
The duty to maintain patient confidences traces back as far as ancient Greece, when the Hippocratic oath established a duty of confidentiality for the medical profession. ("Whatever, in connection with my professional practice, or not in connection with it, I see or hear in the live of men, which ought not be spoken of abroad, I will not divulge as reckoning that all such should be kept secret.")
Modern medical practice continued this ethical obligation of confidentiality through the American Medical Association’s Code of Medical Ethics, which provides that a "physician shall respect the rights of patients – and shall safeguard patient confidences within the constraints of the law." (Preamble Part IV, June 2001).
The notion of confidentiality is grounded in encouraging full uninhibited disclosure by the patient to receive proper treatment and to fulfill the purpose of the doctor/patient relationship. Over time, that has extended to medical professionals surrounding the doctor who must accomplish the patient’s medical goals. Violating these confidences has led to lawsuits brought on common law invasion of privacy grounds, breach of the fiduciary relationship between doctor and patient, intentional and negligent infliction of emotional distress, medical malpractice, defamation, libel, slander, and even breach of express or implied contract theories.
Certainly, a patient’s personal medical information should be controlled by the patient. In holding for the patient, as in Bagent, the court sends a clear message that patient confidentiality must be taken seriously by both employer and employees, particularly in t his age of electronically stored and managed data.
Confidentiality of Medical Information
The subject of this paper is theconfidentiality of medical recordinformation. This is a concern for manypersons, including myself, when visiting ahealth professional. My goal is to talk aboutsome of the issues involved in protectingmedical information, particularly in theelectronic age. I also intend to inform youof recent state (Virginia) and federal effortsto regulate the transmission of medicalinformation. Finally, I plan to share a fewanecdotes and observations regardingcurrent legislative efforts to protect medicalinformation.I recently saw the 1997-1998 premiereof a television program. The program ERis one which many of you may have seen.For those of you who have not, it portraysthe lives of physicians, nurses, and other hospital personnel in a Chicago publicteaching hospital emergency room.
The premiere episode was shot “live” to capture the effect of having cameras recording a documentary in an emergency department.The premiere episode of ER providedmany examples of how medical recordinformation can be released unintentionally,and with significant detrimental impact.Cameras captured images and names ofpatients, and revealed their infections withAIDS and other diseases. Health carepractitioners openly discussed a patient’spersonal history and medical informationon film. These statements of patientidentifying information, if released to thepublic at large, would be breaches ofconfidential information under common law and as codified by various state statutes andfederal regulations.
read more: http://www.mellettepc.com/pdf/fall99.pdf
Malaysian Medical Council “Good Medical Practice”
CONTENTS
1. Principles of Confidentiality
2. Protecting information
3. Sharing information with patients
4. Electronic medical records
5. Disclosure of confidential information with the patient’s consent
6. Disclosure within health care teams
7. Disclosure to employers, insurance companies and other third parties including
managed care organizations
8. Disclosure without the patient’s consent
Disclosure in the patient’s medical interests
Disclosure in relation to the treatment sought by children or others who lack
capacity to give consent
Disclosures where a patient may be a victim of neglect or abuse
Disclosure after a patient’s death
9. Disclosure for medical teaching, medical research, and medical audit
10. Disclosure in the interest of others
Disclosures in the public interest
Disclosures to protect others
11. Disclosure in connection with judicial or other statutory proceedings
Disclosures required by law
Disclosures in the courts
Disclosure to statutory regulatory bodies
12. Media inquiries about patients
Privacy Issues and Medical Practice (Ross Boswell)
Ross Boswell
At the Privacy Issues Forum, Wellington, 30 March 2006
It is a basic tenet of medical practice that information provided to a doctor by a patient must be held in confidence. The applicable item in the New Zealand Medical Association (NZMA) Code of Ethics is:
Protect the patient's private information throughout his/her lifetime and following death, unless there are overriding public interest considerations at stake, or a patient's own safety requires a breach of confidentiality.
Because clinical practice is essentially a collegial endeavour, and because a great deal of medical learning is case-based, this requirement has usually been interpreted as a requirement to keep the information safe within the medical community, rather than safe with one particular doctor. My understanding of the tradition of confidentiality maintained by the legal profession is that, perhaps because of the adversarial basis of legal practice, the legal code is stricter. Legal confidentiality seems require information to be restricted to a particular practitioner rather than within legal community as a whole. The advent of the office of Privacy Commissioner, based on a legal understanding of confidentiality, seems to have brought this difference to the fore. I find I am sometimes surprised by the restrictions proposed and imposed by the legally-trained privacy officers with whom I work in a District Health Board (DHB) environment.
One clear example of this is the medical expectation that an “interesting” case, perhaps an Xray and the brief medical details accompanying it, can and perhaps should be shared informally with other practitioners if there are lessons to be learned from it. The viewpoint taken by privacy officers seems to be that this is equivalent to gossip, and should be discouraged if not prohibited.
The differences are less marked in situations where the sharing of information is for the patient's direct benefit. Under the privacy code, a medical practitioner has not only a right but a duty to share information about a patient with another practitioner providing treatment to that patient. One clear example of this is the sharing of laboratory test results. A patient commonly has tests requested by and reported to a single practitioner. If the results of those tests were available, knowledge of them might prevent another practitioner treating that patient from incurring the delay, inconvenience and cost of repeating the tests.
Since almost all laboratory tests in New Zealand are publicly funded, DHBs which pay for them have an interest in making the results available to prevent expenditure from unnecessary repeat testing. The three Auckland metropolitan DHBs are at present implementing a proposal to have results of all of the tests that they fund, both hospital and community-based, contributed to a computer repository which will be available to accredited hospital clinicians, and in the near future to accredited GPs.
This project has uncovered some interesting privacy issues. It is not possible to predict what test information a patient may consider confidential: for example, while HIV is generally regarded as confidential and cholesterol non-confidential, an HIV test required for an insurance policy may be regarded by a patient as non-confidential, but he may wish to keep his blood cholesterol private to ensure that his wife continues to provide decadent desserts.
The risks of suppressing information may not be apparent to the patient: for example, the fact of attendance at a practitioner (say, at a sexual health clinic) may be regarded by the patient as confidential information to be kept private from his or her GP and other clinicians. If the results of tests from that clinic visit are visible in the repository, then the fact of the visit is disclosed. If they are not visible, then important clues to a subsequent illness may be obscured.
The proposal for the Auckland repository is that all test results will be contributed to the repository, but that patients will be advised of an 0800 phone number they can call to have results from a particular date, or range of dates, suppressed. Their doctors will advise them of this possibility, but also advise them of the potential compromise to best care if they choose to exercise it.
As a final example of a controversial issue of medical privacy, I will briefly describe the debate surrounding the Cervical Screening Amendment Bill of 2004. New Zealand operates a cervical screening register, and women can opt off that register. The Amendment Bill proposed that, in order to ensure that the screening programme can be audited effectively, the medical records of all women diagnosed with cervical cancer must be made available to the audit team. This would hold even for women who had opted off the register. Since cervical cancer is a complication of infection with human papilloma virus, a sexually-transmitted disease, the information required to be provided might include intensely private information such as age at first sexual intercourse, numbers of partners, previous sexually-transmitted infections, pregnancies and terminations of pregnancy.
In commenting on this Bill, the NZMA drew a distinction between public hospital records and private practitioner records. Our view was that it might be argued that public hospital treatment is a public good for which, in return, some compromise in privacy might reasonably be expected. In contrast, private treatment is strictly private, and disclosure of its records for a public screening programme should require explicit patient consent which might, in some cases, be withheld.
In the event, the provisions enacted require practitioners to provide all records on demand, without reference to the patients whose private information they hold, and with draconian penalties (fines of up to $10,000 per day) for failure to comply.
Ross Boswell
Chairman, NZMA
Medical Confidentiality and Legal Privilege
Views and reviews
Medical Confidentiality and Legal Privilege
The fact that the confidentiality of the doctor-patient relationship cannot be guaranteed in a court of law may be news to some patients, but doctors have become injured to the increasing number of occasions on which access to sensitive medical information is demanded and many would expect no better from the courts. The dilemmas presented by HIV disease high-light potential challenges to patient's privacy and expose the inadequate protection offered to it.The recent attention given by the media to several doctor patients with HIV disease caused anger in the medical profession- articulated in the letter pages of this journal- and hammered home the "free for all" that now exists.Should the courts not offer some bulwark to the confidentiality on which patients, even doctor patients, ought to be able to rely? What has happened to the precedent set in the 1988 of X and Y, in which the identity of two general practitioners was prevented from being published in the press by a court injunction? It is difficult not to conclude that medical confidentiality is merely rhetoric, an outdated reminder of the Hippocratic tradition.
In considering how much protection ought to be given to doctor-patient communications Jean McHale assesses whether the absence of legal privilege or, a statute on confidentiality, has fatally undermined medical confidentiality and goes on to make a case for granting such a privilege.
McHale, a lawyer, compares the grounds on which privilege is available to her own profession in order to validate the claims of the medical profession for a similar right. She argues that it would not be justifiable to grant a privilege simply on the grounds that doctors, like lawyers, are professionals and as such need confidentiality for their medical practice.Proof that a lack of privilege hinders doctors' practice is difficult to find. The privilege given to information between client and lawyer is, after all, not the lawyer's privilege but that of a client. Similarly, privilege for medical confidences should exist because we believe that patients have fundamental rights which must be safeguarded. McHale sees the right to stop disclosure as part of the patient's right to privacy. In her carefully argued text she explores the concept of privacy of the individual and the grounds on which a legal privilege might be based, and assesses the situation in the different jurisdictions that already have legislation and considers its practical implications.
The book will widen doctors'perspectives on medical confidentiality through the analysis of its historical development, the carefully selected illustrative court cases, and the interesting selections from medical publications. Seigler's description from the New England Journal of Medicine of a "decrepit concept" prompted by his study of who had access to the medical records of patients in his local hospital seems entirely appropriate. He discovered that the figure was between 75 and 100 medical staff.
The analysis is thorough and the arguments persuasive, but do they convince? Not entirely. The problems raised by a legal privilege for medical information include increased costs of legal process, difficulties in defining the boundaries of the confidential, and the persisting burden of discretion still vested with judges in assessing when the privilege should be overruled or tempered. While most will support McHale's call for scrutiny of the many statutes which currently require disclosure of evidence to ascertain to what extent they are really necessary, I suspect that evidential privilege for medical information may simply replace dissatisfaction with the status quo with a different set of complaints. McHale does not adequately rebut such a conclusion but acknowledges that her proposals are no panacea. Nevertheless, her considered analysis is an excellent starting point to finding some way of preserving the vestiges of medical confidentiality.
Medical Ethics and Law: Confidentiality
confidentiality Hippocrates said: ‘Whatever, in connection with my professional practice or not
in connection with it, I see or hear in the life of men, which ought not to be spoken of abroad, I
will not divulge, as reckoning that all such should be kept secret.’ The obligation of
confidentiality spoken of here is not absolute; it is up to the doctor to decide what information
‘ought not to be spoken abroad.’ Another Oath of confidentiality is the Declaration of Geneva
which says: ‘I will respect the secrets confided in me, even after the patient has died.’ Here,
however, the obligation is absolute. These are two sources of a doctor’s duty of confidentiality
which, although they differ in extent, both highlight the importance of respecting the
confidentiality of patients. J NI Ethics Forum 2006, 3: 146-153
Download Good Medical Practice
Confidentiality case studies
Case 1 | ||||||
Your 36-year-old patient has just tested positive for HIV. He asks that you not inform his wife of the results and claims he is not ready to tell her yet. What is your role legally? What would you say to your patient? How is this case different from Case 1? How would you handle this situation? Would you tell his wife? What are you required to do legally? |
Confidentiality - ETHICS IN MEDICINE
Good Medical Practice (2006)
About Good Medical Practice
Good Medical Practice sets out the principles and values on which good practice is founded; these principles together describe medical professionalism in action. The guidance is addressed to doctors, but it is also intended to let the public know what they can expect from doctors.
We have provided links to other guidance and information which illustrate how the principles in Good Medical Practice apply in practice, and how they may be interpreted in other contexts; for example, in undergraduate education, in revalidation, or in our consideration of a doctor's conduct, performance or health through our fitness to practise procedures. There are links to:
- supplementary guidance and other information from the GMC
- cases heard by GMC fitness to practise panels, which provide examples of where a failure to follow the guidance in Good Medical Practice has put a doctor's registration at risk
- external (non-GMC) sources of advice and information.
Using Good Medical Practice on-line
- To read the guidance as it appears in the print version, you can use the navigation arrows at the bottom of the page. Begin reading.
- For an overview of Good Medical Practice you can use the contents page.
- You can use the links in the boxes on the right to access ‘Further GMC guidance’ and ‘External web links’. Where a link refers to a specific paragraph or bullet point, this is indicated using square brackets (for example [4d] Consent guidance).
- You can search all our current ethical guidance, including Good Medical Practice, using our A-Z of Ethical Guidance.
Preserving confidentiality in occupational medical practice
American Family Physician, April, 1996 by Robert J. McCunney
E-MAILPRINTLINK
The physician's challenge in preserving medical confidentiality derives primarily from his or her position between patients on the one side and businesses, regulatory agencies and insurance carriers on the other side, each of whom have a vested interest in patients' medical information. In providing occupational medical services, the family physician is faced with requests for confidential information from a variety of sources.[1,2] Despite the lack of a uniform national law regarding medical confidentiality, various codes of conduct and relevant parts of the Americans with Disabilities Act (ADA) of 1992 can help physicians make informed decisions about requests for medical information. In all codes of conduct, the patient decides whether and when to release such material, unless overruled by public health risks or laws. This article provides a framework for response to requests for information by reviewing several codes of conduct and legislation affecting confidential medical information and applying them to five clinical care situations.
Sunday, October 19, 2008
The Complete Guide to Medical Writing By Mark C. Stuart
Saturday, October 18, 2008
WHO- Postgraduate course 2007 – Cohort studies February 28, 2007
WHO- Postgraduate course 2007 – Cohort studies February 28, 2007
from:http://www.gfmer.ch/Medical_education_En/PGC_RH_2007/pdf/Cohort_studies_Wolff_WHO_2007.pdf
Strategies for data analysis: Cohort studies
Strategies for data analysis: Cohort studies
From research to practice: Postgraduate training in reproductive health/chronic disease
Dr Lale Say
March 2003
from: http://www.gfmer.ch/Endo/Course2003/PDF/Cohort_studies_Say.pdf
Friday, October 17, 2008
Smokingin early adolescence : 1993 Pelotas birth cohort study
Cohort Profile: The 1982 Pelotas (Brazil) Birth Cohort Study
IJE Advance Access originally published online on December 22, 2005
International Journal of Epidemiology 2006 35(2):237-242; doi:10.1093/ije/dyi290
|
Cohort Profile |
Cohort Profile: The 1982 Pelotas (Brazil) Birth Cohort Study
1 Programa de Pós-Graduação em Epidemiologia, Universidade Federal de Pelotas, Brasil
2 PAHO/WHO Latin American Centre for Perinatology, Montevideo, Uruguay
* Corresponding author. Avenida Duque de Caxias, 250, CEP: 96030-002, Pelotas, Rio Grande do Sul, Brasil. E-mail: cvictora@terra.com.br
How did the study come about? |
---|
Top How did the study... What does the study... Who is in the... How often have they... What has been measured? What has it found? What are the main... Can I get hold... Supplementary Data References |
---|
Pelotas is a city in the extreme south of Brazil, near the border of Uruguay, with 214 000 urban inhabitants in 1982. At the time, we were assistant professors, each working in one of the two medical schools in the city, and both undergoing post-graduate training at the University of London. We were inspired by the findings of the British perinatal study, and one of us (FCB) decided to do a similar study for his doctoral thesis. The lack of reliable data on perinatal mortality in Brazil, due to poor registration of births and deaths—particularly stillbirths—justified the launch of the study. Funding from the International Development Research Center (Canada) was obtained for the perinatal survey, which led to FCB's PhD thesis.1,2
While perinatal data collection was underway, we obtained a grant to visit a sub-sample of these newborns at the age of12 months. Later funds were obtained for two visits to the entire cohort, at the approximate ages of 2 and 4 years.
Little did we imagine that our study would eventually become one of the largest and longest running birth cohorts in thedeveloping world.3
What does the study cover? (and how has this changed) |
---|
Top How did the study... What does the study... Who is in the... How often have they... What has been measured? What has it found? What are the main... Can I get hold... Supplementary Data References |
---|
Initially, the study focus was on perinatal, infant, and early childhood morbidity and mortality. We were particularly interested in breastfeeding patterns and nutritional status, as well as social and environmental factors. Deaths of cohort members were identified by regular visits to all hospitals, cemeteries, offices of civil registrations, and local health authorities, since 1982.
By mid-childhood, the study shifted in emphasis to child care, utilization of health services, selected morbidity indicators, and child development. A random sub-sample of 360 four-year-olds was selected for an in-depth study of psychomotor development.
In adolescence, issues related to sexual and reproductive behaviours (including teenage pregnancies), habits such as smoking and alcohol drinking, mental health, and education became the focus of the investigation. A sub-study investigated oral health in a random sample of 900 adolescents, and an ethnographic study of 96 cohort members, stratified by sex and socioeconomic status, has included repeated in-depth visits from the age of 15 to 23 years, aimed at understanding the role of adolescent development in influencing high-risk behaviours.
In more recent phases, with cohort members being young adults, the main emphasis has shifted to risk factors for chronic disease (including smoking, diet, physical exercise, and overweight), reproductive history, and mental health.
Who is in the sample? |
---|
Top How did the study... What does the study... Who is in the... How often have they... What has been measured? What has it found? What are the main... Can I get hold... Supplementary Data References |
---|
During the whole of 1982, the three maternity hospitals in the city were visited daily and 7392 births were recorded. Of these, 6011 infants were born to mothers living in the urban area of Pelotas. Using data from birth registration and from a city census, we identified another 46 children who were delivered at home in 1982, so that our hospital sample accounts for 99.2% of all births in the city. The 5914 live born infants constitute our original cohort.
Brazil is a country with wide social disparities, and a population-based sample covering the entire social spectrum allows the detailed study of long-term consequences of poverty on health.
How often have they been followed-up, and what was attrition like? |
---|
Top How did the study... What does the study... Who is in the... How often have they... What has been measured? What has it found? What are the main... Can I get hold... Supplementary Data References |
---|
Follow-up procedures have been somewhat haphazard, depending on the availability of funding. Table 1 summarizes the main visits to the cohort. Earlier publications provide further details about each visit.4,5
|
In early 1983, the available funding was barely sufficient for visiting one-third of the cohort children. We opted for examining those born from January to April 1982, who were 1 year old at the time. This visit was funded by the World Health Organization. Using the addresses available from hospital records, we examined 1457 children who, added to those known to have died, comprised 80% of the target group (Table 1).
We became quite worried about the loss of one-fifth of the cohort in a single year, and decided to change the search strategy in the next visits. The 1984 and 1986 visits were planned well in advance, and received substantial funding from the United Kingdom's Department for International Development (then known as Overseas Development Administration). The objective was to examine every cohort child who was still living in the urban area. Rather than relying on addresses, in early 1984 we decided to visit every household in the city in search of children born in 1982. This led to 87% of the original cohort being traced, a substantial improvement over the follow-up rate obtained a year earlier. Of those located, 45% of families had already changed addresses since the cohort child had been born, indicating a very high rate of mobility. The same approach was used in early 1986, leading to a follow-up rate of 84%.
At that stage, we had a massive amount of data to analyse as a small research unit. Because our primary interest was in child health, we assumed that the cohort study had been completed and set out to write up our results and become involved in other studies.
There was no new data collection until 1995, when we were approached by the United Nations Children's Fund (UNICEF) and Development Fund for Women (UNIFEM) to collect information on issues related to adolescent sexuality. Limited funds were available, and the addresses obtained in 1986 were used to trace a random sample of 1100 cohort members. Only 70% of them could be located.
This visit renewed our interest in the cohort, and in 1997 we decided to apply a similar approach to that used in the citycensuses of 1984 and 1986. As funds—this time from the Brazilian government—were not available to cover the wholecity, we systematically selected 70 census tracts (27% of the total) and visited every household in those tracts. This led to 72% of the cohort members expected to be living in those tracts to be traced.
A special opportunity for follow-up was provided by the compulsory Army recruitment examination, held in 2000. All cohort males who were still living in the city were legally required to attend a local Army base to undergo a physical examination in August and September. Our research team was deployed to the base and was able to examine 79% of males from the cohort. Funds from the Brazilian government supported this examination.
2001 marked the end of adolescence for the cohort. We had special interest in investigating the high rates of teenage childbearing in the cohort, and obtained funding from the World Health Organization for this purpose. Using the national computerized birth registration system put in place in the late 1990s, we identified over 400 of cohort women who had delivered an infant up to March 2001, and visited them at home. To obtain a comparison group for a case–control analysis, we revisited all households in the 70 census tracts from the 1997 sample and examined all cohort subjects living in these tracts. An extensive interview was carried out with women and a shorter version with men, who had already been examined in the Army in 2000. The follow-up rate was 69%.
At this time, it was evident that we had the largest running birth cohort outside high-income countries, and our interest was renewed. We were able to obtain substantial funding from the Wellcome Trust to visit the whole cohort once again. FromOctober 2004 to August 2005, we visited all 98 000 households in the city and located 3924 cohort members. The system for monitoring mortality had identified 282 deaths. For those who had not been located and were not known to have died, we used the last known address and existing databases (including universities, secondary schools, telephone directories) for another attempt. This allowed us to interview 4297 subjects. Added to those known to have died, these represent a follow-up rate of 77% in relation to the original cohort.
We have carried out several analyses of attrition rates according to baseline characteristics.5 The pattern of follow-up varies slightly according to the search strategy used, but in general subjects born to middle-class families are easier to trace than those born in either the upper or lower ends of the social distribution. There are no consistent differences in follow-up according to sex, birthweight, or skin colour. Subjects born to unmarried mothers are consistently harder to trace than those born in wedlock.
What has been measured? |
---|
Top How did the study... What does the study... Who is in the... How often have they... What has been measured? What has it found? What are the main... Can I get hold... Supplementary Data References |
---|
The 1982 questionnaire was extremely short. Owing to our lack of experience, we worried about having several sheets of paper that might become separated and decided to used the longest sheet of paper commercially available (a bit longer than A4 size) and restrict the questionnaire to both sides of this sheet. All the information contained in this form took 80 columns in a punch card, which also made it rather convenient.
Table 2 shows the main categories of variables collected in the early phases of the study. In the hospital interview, mothers answered questions on socioeconomic, demographic, and health-related variables. Their infants were weighed with regularly calibrated paediatric scales (Filizolla, Brazil) to the nearest 10 g. Birth length was not recorded. Mothers were weighed and measured.
|
As we grew more confident, our questionnaires grew longer—possibly too much so—and the examinations more thorough. Table 3 shows the main categories of variables included in recent visits. The questionnaires now include two forms, one applied by an interviewer and another self-applied confidential form that is identified only by a questionnaire number.
|
In 2000, biological materials were collected from males (blood samples from which sera were extracted and frozen at –70°C) and in 2005 for both sexes (extracted DNA samples as well as sera). In 2000, males were weighed with an electronic Tanita Body Fat Analyzer scale (model TBF-305; Tokyo, Japan), which also provided information on body composition through bio-impedance. These results were validated in a sub-sample of 48 subjects by comparison with total body water estimated through isotopic methods.6
Ethical requirements evolved considerably during the study period. In the early phases, verbal consent was obtained, and there were no local ethical review committees. Recent phases comply with current requirements of ethical review and include written informed consent. Special provisions are made for the ethical use of biological materials.
What has it found? |
---|
Top How did the study... What does the study... Who is in the... How often have they... What has been measured? What has it found? What are the main... Can I get hold... Supplementary Data References |
---|
A full list of all publications to date from the study is available as supplementary data at IJE Online.
The first publications from the cohort addressed perinatal issues, highlighting the magnitude of the problems of perinatal mortality and low birthweight, and describing risk factors for these outcomes.1,7,8 The extremely high rate of caesarean sections (28%) was alsohighlighted in the publications,9 as were infant mortality levels, causes, and risk factors.10–12
Infant feeding patterns and their influence on health has been a major theme in our study.13–16 Given the prospective nature of our data, it has been possible to avoid some of the biases arising from using recalled breastfeeding duration in retrospective cohorts, and to investigate the long-term effects of breastfeeding on health and educational attainment, while adjusting for several early life factors.17–19
During the childhood phase, we investigated issues related to malnutrition and infection,20,21 as well as the associationsbetween health and nutrition outcomes with maternal education22 and birth spacing.23
A cross-cutting theme in our cohort study has been the effect of social inequalities on health.24 In 1988, our book entitled‘Epidemiology of Inequality’ came out in Portuguese, addressing social differentials in terms of perinatal, infant, and childhood outcomes.25 Five thousand copies were sold, and the Spanish version was published by the Pan-American Health Organization in 1992.
More recent publications address issues of adolescent health, including the current epidemics of overweight,26 adolescentpregnancy,27 and asthma.28
Much of our current work is aimed at investigating the effects of low birthweight (which we can separate into intrauterinegrowth restriction and preterm delivery) and growth in childhood on several outcomes. These include blood pressure,29overweight,26 and lung function.28 Analyses are forthcoming on blood lipids, glycaemia, body composition, oral health, and educational achievement.
What are the main strengths and weaknesses? |
---|
Top How did the study... What does the study... Who is in the... How often have they... What has been measured? What has it found? What are the main... Can I get hold... Supplementary Data References |
---|
Cohorts from low-income and middle-income countries are needed because findings are likely to differ from those obtained in developed countries. Some exposures may have different characteristics—for example, most physical activity among males in our cohort isfrom occupation, rather than from leisure time activities. Another example is breastfeeding—whereas non-breastfed babies in high-income countries are likely to receive infant formula, in our setting cow's milk is the main breast milk substitute so the results of comparisons of breastfed and artificially fed infants may differ between populations. Also, some exposures—for example low birthweight or childhood malnutrition—are much more common than in developed countries, and their long-term effects can be studied with greater precision. Finally, confounding factors may act in different directions in rich and poor countries. For example, while breastfeeding tends to be associated with high socioeconomic status in wealthy populations, the reverse is often the case in low-income and middle-income countries. As a consequence, residual confounding—a critical issue in the study of the effects of breastfeeding on adult health—may operate in different directions.
A review by Harpman et al.3 identified our study as the largest and longest running prospective birth cohort study in a developing country. About 4000 variables are available for subjects seen in all phases of the study, including anthropometric measurements not only at birth but also at different ages in childhood. Other strengths include the population base and relatively high follow-up rates. Although the latter are considerably lower than some of the studies from developed countries, we have had to face the challenge of tracing people actively, rather than passively through national databases.
Our success is largely related to the characteristics of the city. Pelotas is a middle-size city with relatively low rates of in-migration and out-migration. The number of annual births provides sufficient statistical power for the study, while being still logistically manageable. Also, concerns with personal and home security are not as manifest as in larger Latin Americancities and refusals are rare.
On the negative side, there are many things that we—given the benefit of hindsight—would have done differently. At the perinatal interview, we should have measured birth length and assessed gestational age through physical examination (20% of the mothers were unable to recall the dates of their last menstrual period). We collected data on family income as a grouped variable rather than recording it as a continuous variable.
In retrospect, obtaining information on the whole cohort at a smaller number of visits would probably have been better than using sub-samples. To date, we have data on the entire cohort for the original perinatal interview, the 1984, 1986, and 2004–05 follow-up visits, as well as for the 2000 Army examination for males. The 1983, 1995, 1997, and 2001 visits were based on sub-samples, with different sampling approaches. This means that only a few hundred subjects have complete data from all follow-up visits.
Some lessons were learned from the Pelotas cohort that may be relevant to other studies. A critical issue in all cohorts isthat of attrition. Two successful strategies were used in our study: household sampling and Army enlistment. The first entailed visiting all, or a sample of, the city's households to identify individuals born in 1982, and later tracing them to their cohort records. The second included taking advantage of the compulsory Army enlistment process. Attempts at locating cohort members using available addresses, both in 1983 and in 1995, led to high rates of attrition.
Lessons can also be learned regarding administrative and financial aspects of the study. Funding agency fatigue means that few are prepared to support more than one to two rounds of the study. This precluded a more regular schedule of visits, and sampling fractions were sometimes determined by availability of funds rather than by scientific principles, as for example in the 1983 and 1995 follow-ups. Large birth cohort studies present specific funding issues that should deserve special treatment by grant-making agencies.
As cohort members reach adult age, it is likely that they will increasingly move out of the city, where job opportunities arescarce. This may lead to higher attrition in the near future. Alternative approaches will have to be conceived, includingpassive follow-up through death certificates and nested case–control studies, among others. Nevertheless, the large amount of data already available from the study will certainly lead to many additional analyses on health, behaviour, and development in childhood and adolescence.
Eleven years after the first cohort, we started another study—the 1993 Pelotas Birth Cohort—including all births in thecity. The original plan was to launch it in 1992, a decade later than the first study, but of course funding was delayed. Thisstudy is underway and 87% of them were traced at the age of 12 years. In 2004, again 11 years later, the 2004 birth cohort was launched. The existence of three parallel birth cohorts is allowing a unique comparison of secular trends in child health, and in the future in adolescent and adult health.30
Research groups involved in cohort studies have long realized that it is difficult, if not impossible, to stop the studies while at least some of the participants are still alive. We are finding it hard not only to stop our ongoing cohorts but also to refrain from starting a new one every 11 years. Watch this space for the 2015 Pelotas Birth Cohort Study!
Can I get hold of the data? Where can I find more? |
---|
Top How did the study... What does the study... Who is in the... How often have they... What has been measured? What has it found? What are the main... Can I get hold... Supplementary Data References |
---|
We welcome joint analyses of the cohort data. We have collaborated successfully with investigators from the UK (London School of Hygiene and Tropical Medicine, Institute of Child Health), USA (Cornell, Emory and Michigan Universities) and Australia (Universityof Newcastle) as well as several Brazilian institutions. Our most fruitful experiences have been from having doctoral or post-doctoral fellows come to Pelotas for a few months at atime, to analyse data and write.
For interested young researchers from Latin America, we launched a Wellcome-Trust sponsored post-graduate programme in Life Course Epidemiology in 2005, which now includes six MSc and PhD students from the region who receive full scholarships to work on our cohorts. Applications are welcome.
For further information contact our website at http://www.epidemio-ufpel.org.br/projetos_de_pesquisas/estudos/coorte_1982 or e-mail cvictora@terra.com.br.
Supplementary Data |
---|
Top How did the study... What does the study... Who is in the... How often have they... What has been measured? What has it found? What are the main... Can I get hold... Supplementary Data References |
---|
Supplementary data are available at IJE Online.
Acknowledgments |
---|
This analysis was supported by the Wellcome Trusts initiative entitled Major Awards for Latin America on Health Consequences of Population Change. Earlier phases of the 1982 cohort study were funded by the International Development Research Center (Canada), the World Health Organization (Department of Child and Adolescent Health and Development, and Human Reproduction Programme), the Overseas Development Administration (UK), the United Nations Development Fund for Women, the National Program for Centers of Excellence (Brazil), the National Research Council (Brazil), and the Ministry of Health (Brazil). Special thanks to Patrick Vaughan who helped us launch the study in its early phases. We would also like to thank our many colleagues who participated in the several phases of the study, particularly Ana Maria Borges Teixeira, Angela Maas, Angela Oliveira, Ari Lemos Jr, Aydin Nazmi, Beatriz Guimarães, Bernardo Horta, Celene Longo da Silva, Cintia Lombardi Nash, Cora Araújo, Denise Gigante, Dominique Behague, Gicele Minten, Helen Gonçalves, Isabel Oliveira, Jorge Umberto Béria, José Carlos Martines, Luciana Anselmi, Magda Damiani, Mario Azevedo Jr, Paulo Orlando Monteiro, Paulo Post, Pedro Olinto, Rosângela Lima, Sharon Huttly, and Vera Silveira.
References |
---|
Top How did the study... What does the study... Who is in the... How often have they... What has been measured? What has it found? What are the main... Can I get hold... Supplementary Data References |
---|
1 Barros FC, Victora CG, Vaughan JP, Estanislau HJ. Perinatal mortality in southern Brazil: a population-based study of 7392 births. Bull World Health Organ 1987;65:95–104.[ISI][Medline]
2 Barros F. The Epidemiology of Perinatal Health in Southern Brazil: A Study of Perinatal Mortality, Low Birth Weight and the Utilisation of Health Care. London, UK: University of London, 1985.
3 Harpham T, Huttly S, Wilson I, DeWet T. Linking public issues with private troubles: panel studies in developing countries. J Int Dev 2003;15:353–63.[CrossRef]
4 Victora CG, Barros FC, Martines JC, Beria JU, Vaughan JP. Longitudinal study of children born in Pelotas, RS, Brazil in 1982. Methodology and preliminary results. Rev Saude Publica 1985;19:58–68.[ISI][Medline]
5 Victora CG, Barros FC, Lima RC et al. The Pelotas birth cohort study, Rio Grande do Sul, Brazil, 1982–2001. Cad Saude Publica 2003;19:1241–56.[Medline]
6 Wells JCK, Gigante DP, Wright A, Hallal PC, Victora CG. Validation of leg-to-leg impedance for body composition assessment among Brazilian males aged 16–19 years. Int J Body Composition Res 2003;1:1–6.[CrossRef]
7 Barros FC, Victora CG, Vaughan JP, Jair Estanislau H. Low-birth weight in the municipality of Pelotas, Brazil: risk factors. Bol Oficina Sanit Panam 1987;102:541–54.[Medline]
8 Barros FC, Huttly SR, Victora CG, Kirkwood BR, Vaughan JP. Comparison of the causes and consequences of prematurity and intrauterine growth retardation: a longitudinal study in southern Brazil. Pediatrics 1992;90:238–44.
9 Barros FC, Vaughan JP, Victora CG, Huttly SR. Epidemic of caesarean sections in Brazil. Lancet 1991;338:167–69.[CrossRef][ISI][Medline]
10 Barros FC, Victora CG, Vaughan JP, Teixeira AM, Ashworth A. Infant mortality in southern Brazil: a population based study of causes of death. Arch Dis Child 1987;62:487–90.[Abstract]
11 Victora CG, Barros FC, Vaughan JP, Teixeira AM. Birthweight and infant mortality: a longitudinal study of 5914 Brazilian children. Int J Epidemiol 1987;16:239–45.
12 Victora CG, Barros FC, Huttly SR, Teixeira AM, Vaughan JP. Early childhood mortality in a Brazilian cohort: the roles of birthweight and socioeconomic status. Int J Epidemiol 1992;21:911–15.
13 Barros FC, Victora CG, Vaughan JP, Smith PG. Birth weight and duration of breast-feeding: are the beneficial effects of human milk being overestimated? Pediatrics 1986;78:656–61.
14 Barros FC, Victora CG, Vaughan JP. Breastfeeding and socioeconomic status in Southern Brazil. Acta Paediatr Scand 1986;75:558–62.[ISI][Medline]
15 Victora CG, Huttly SR, Barros FC, Vaughan JP. Caesarean section and duration of breast feeding among Brazilians.Arch Dis Child 1990;65:632–34.[Abstract]
16 Victora CG, Huttly SR, Barros FC, Martines JC, Vaughan JP. Prolonged breastfeeding and malnutrition: confounding and effect modification in a Brazilian cohort study. Epidemiology 1991;2:175–81.[Medline]
17 da Costa Lima R, Victora CG, Menezes AM, Barros FC. Do risk factors for childhood infections and malnutrition protect against asthma? A study of Brazilian male adolescents. Am J Public Health 2003;93:1858–64.
18 Victora CG, Barros F, Lima RC, Horta BL, Wells J. Anthropometry and body composition of 18 year old men according to duration of breast feeding: birth cohort study from Brazil. BMJ 2003;327:901–04.
19 Victora C, Barros F, Horta B, Lima R. Breastfeeding and school achievement in Brazilian adolescents. Acta Paediatrica 2005;94:1656–60.[CrossRef][ISI][Medline]
20 Victora CG, Barros FC, Kirkwood BR, Vaughan JP. Pneumonia, diarrhoea, and growth in the first 4 y of life: a longitudinal study of 5914 urban Brazilian children. Am J Clin Nutr 1990;52:391–96.
21 Huttly SR, Victora CG, Barros FC, Teixeira AM, Vaughan JP. The timing of nutritional status determination: implications for interventions and growth monitoring. Eur J Clin Nutr 1991;45:85–95.[ISI][Medline]
22 Victora CG, Huttly SR, Barros FC, Lombardi C, Vaughan JP. Maternal education in relation to early and late child health outcomes: findings from a Brazilian cohort study. Soc Sci Med 1992;34:899–905.[CrossRef][ISI][Medline]
23 Huttly SR, Victora CG, Barros FC, Vaughan JP. Birth spacing and child health in urban Brazilian children. Pediatrics1992;89:1049–54.
24 Victora CG, Vaughan JP, Barros FC, Silva AC, Tomasi E. Explaining trends in inequities: evidence from Brazilian child health studies. Lancet 2000;356:1093–98.[CrossRef][ISI][Medline]
25 Victora CG, Barros FC, Vaughan JP. Epidemiologia da desigualdade: um estudo longitudinal de 6000 crianças brasileiras. São Paulo: HUCITEC, 1989.[CrossRef]
26 Monteiro PO, Victora CG, Barros FC, Monteiro LM. Birth size, early childhood growth, and adolescent obesity in a Brazilian birth cohort. Int J Obes Relat Metab Disord 2003;27:1274–82.[CrossRef][ISI][Medline]
27 Gigante DP, Victora CG, Gonçalves H, Lima RC, Barros FC, Rasmussen KM. Risk factors for childbearing during adolescence in a population-based birth cohort in southern Brazil. Rev Panam Salud Publica 2004;16:1–10.[ISI][Medline]
28 Lima Rda C, Victora CG, Menezes AM, Barros FC. Respiratory function in adolescence in relation to low birth weight, preterm delivery, and intrauterine growth restriction. Chest 2005;128:2400–07.
29 Barros FC, Victora CG. Increased blood pressure in adolescents who were small for gestational age at birth: a cohort study in Brazil. Int J Epidemiol 1999;28:676–81.
30 Barros FC, Victora CG, Barros AJ et al. The challenge of reducing neonatal mortality in middle-income countries: findings from three Brazilian birth cohorts in 1982, 1993, and 2004. Lancet 2005;365:847–54.[CrossRef][ISI][Medline]
CiteULike Connotea Del.icio.us What's this?
This article has been cited by other articles:
C. G Victora, P. C Hallal, C. L. Araujo, A. M. Menezes, J. C. Wells, and F. C Barros Cohort Profile: The 1993 Pelotas (Brazil) Birth Cohort Study Int. J. Epidemiol., August 1, 2008; 37(4): 704 - 709. [Full Text] [PDF] | |||||
L. M. Richter Studying adolescence. Science, June 30, 2006; 312(5782): 1902 - 1905. [Abstract] [Full Text] [PDF] | |||||
|