- EpiData-list - University of Manitoba Mailing Lists

Epidata XML to R: how to handle labels?
by epidata-list＠lists.umanitoba.ca 14 Jun '11

14 Jun '11

I would like to hear opinions about using epidata labels when importing into R. Here's a fragment of XML showing a value label set from sample.epx that comes with the data manager. <ValueLabelSet id="valuelabelset_id_1"> <Type>3</Type> <Name>Float Set</Name> <Internal> <ValueLabel order="1" value="1,11"> <Label lang="en">Float value A</Label> </ValueLabel> <ValueLabel order="2" value="2,22"> <Label lang="en">Float value B</Label> </ValueLabel> <ValueLabel order="3" value="3,33"> <Label lang="en">Float value C</Label> </ValueLabel> <ValueLabel order="4" value="8,88" missing="true"> <Label lang="en">Second last missing</Label> </ValueLabel> <ValueLabel order="5" value="9,99" missing="true"> <Label lang="en">Float missing</Label> </ValueLabel> </Internal> </ValueLabelSet> This says that a numeric (float) variable that uses this label set should give the numeric value 1.11 the label "Float value A", etc. and defines two types of missing value: 8.88 and 9.99. R does not have a concept of labelling values in this way (unless things have changed recently; I have not kept up with the R list for a couple of years or so). R also does not have the concept of multiple missing values, you either have a value to work with, or you don't. It is possible to attach a comment to an R object, but these are not transferred when an object is copied, so they can easily become lost. I think we have the following options: 1) use the actual stored value (e.g. 1.11), ignore the labels except for the missing value definitions and for these just make them NA. This loses information that might be useful, but means that numeric values (and dates, etc) keep their data type and can be analysed appropriately, i.e. it is possible to calculate the mean of 1.11, but not the mean of "Float value A"; 2) use the labels in all cases. The result of this is the opposite of option 1: we keep the extra information that the labels provide, but coerce all data to characters/factors and can no longer do many statistical analyses; 3) create a new column for each variable that uses a label set, so that we have both the original data and next to it a column with the labels. This could result in a much larger data set in R, and possible confusion, especially if analyses change the value of one column and not the other. 4) Code in options 1-3, allowing the user to specify which approach to take; 5) Code in options 1-3, allowing the user to specify for each column which approach to take. 6) Do nothing at all to the data, just return the label information as part of the result of the read function (in the same way that the table structure is returned and the study meta data will be returned) so that the user can then easily read it in R and use it to recode variables/values appropriately. This would mean manually recoding missing values. I could write functions that are able to work with this information so that users can then fairly easily query the label set representation to find out the label information. In R this could work like this, assuming that x is an object that contains label information: epidata.label.value(1.11, x, "Float Set") > "Float value A" epidata.label.value(8.88, x, "Float Set") > "Second last missing" ## If the value is not in the value set, return NA epidata.label.value(1.66, x, "Float Set") > NA is.epidata.label.na(1.11, x, "Float Set") > FALSE is.epidata.label.na(8.88, x, "Float Set") > TRUE is.epidata.label.na(1.66, x, "Float Set") > NA As I work through this I am tending towards option 6; I would probably have to write functions like this anyway, and I think that making them visible to the user provides the greatest flexibility and probably limits the risk of being surprised. Recoding a variable in R using the labels would then work like this, assuming that x is an object created by importing an epidata XML file into R, and that the dataframe contains a field called "height": ## Extract the data frame and labels into new objects to make the code easier to read y <- x[[1]] z <- x[["value.labels"]] ## Apply the labels y$height <- epidata.label.value(y$height, z, "Float Set") ## Set the missing values is.epidata.label.na(y$height, z, "Float Set") <- NA David -- David Whiting, PhD | Senior Epidemiology & Public Health Specialist tel +32-2-6437945 | mob +32-496-266436 | David.Whiting(a)idf.org International Diabetes Federation 166 Chaussée de La Hulpe, B-1170 Brussels, Belgium tel +32-2-5385511 | fax +32-2-5385114 info(a)idf.org | www.idf.org | VAT BE 0433.674.528 IDF | Promoting diabetes care, prevention and a cure worldwide

1 6

question about double entry
by epidata-list＠lists.umanitoba.ca 14 Jun '11

14 Jun '11

Hello My question is if there are any possibilities to check up for double entry in EPIDATA ? Susann R Susann Regber Pediatric nurse, MPH, PHD-student Nordic School of Public Health Tel.: +46 (0)31 69 39 01 Mobile: +46(0) 738 19 04 19 E-mail: susann.regber(a)nhv.se Homepage: www.nhv.se P.O. Box 12133, SE-402 42 Gothenburg

1 1

Value labels and variable labels in R
by epidata-list＠lists.umanitoba.ca 14 Jun '11

14 Jun '11

Hello EpiData (Dungeons & Dragons :-D) Masters, I did read this post but after a second reply the messages get a little confused. So please forgive me if Im a little repetitive. I see at least three types of labels: 1. Data label 2. Variable label 3. Value label Im not aware of any data label function in R. In R, the variable label is the variable name, which is the charater vector on the top of the columns. So, if you have a dataset on object *data* and type > labels(data) # It will returns the columns and rows names > colnames(data) # will return the same columns names as labels. Value labels does not make sense in R, because labels are a discription of numeric values which represent categories. Thus, these numbers are not really numbers, rather categories. Most importing functions transforms these into factors, which will make sense in most analysis functions. What are value labels for? If you run a table analysis the labels comes in place of 1 or 2 or 99 or whatever values the labels should represent. R does it through the factors and their levels. What are the variable labels for? If you run a table analysis the labels comes in marginals in place of the variables names (in this case as dimnames()) that could be something meaningless such as v22. However the table and other functions in R do not recognize labels wherever they are. Thus importing variable labels would be of little use. Im awere of two packages that have importing functions that bring together with the data the variable labels and transform the value labels into factors, and uses these variables labels at the analysis functions automatically. epicalc and rms. epicalc does through the label.var()functionand Im not really sure where the label is stored. rms (or perhaps Hmisc... not sure) package uses the label() function - not the labels() function - and store the labels as attributes of each variable. If you type for example > attr(data$v22) # the label should appear as one of the variable attributes such as 'Age' or 'Sex'. Both packages uses these functions throughout all analysis functions and have functions to print a map of labels such as describe() and desc(). Describe is more rich and also returns some summaries of the variables. rms also has a datadist() function which is able to store at the R option() the distributions of variables values, which may be useful to set potential values in analysis tha your data is not representing (such as a bar plot that should range from 10 to 80 but your lowest and highest values are 12 and 68). But see, these variable labels will only work on analysis functions of epicalc if I use the importing function from epicalc - use() - and for rms if I use the rms importing functions - stata.get(), spss.get(), sas.get(). If I import with stata.get for example and run analysis with any other package than rms, the labels are ignored. :-( At the moment, I cant see how these can be improved unless a package EpiData comes along with analysis functions that automatically call the variable labels on tables and graphs. If variables labels are to be imported then storing them as variable attributes seems to be reasonable, instead of storing them at a second column in the same dataframe. But the way I see, the next step (analysis functions to be used) is the one that should guide how labels should be handled. Kind regards, Abraço forte e que a força esteja com você, Dr. Pedro Emmanuel A. A. do Brasil Instituto de Pesquisa Clínica Evandro Chagas Fundação Oswaldo Cruz Rio de Janeiro - Brasil Av. Brasil 4365 Tel 55 21 3865-9648 email: pedro.brasil(a)ipec.fiocruz.br email: emmanuel.brasil(a)gmail.com ---Apoio aos softwares livres www.zotero.org - gerenciamento de referências bibliográficas. www.broffice.org ou www.openoffice.org - textos, planilhas ou apresentações. www.epidata.dk - entrada de dados. www.r-project.org - análise de dados. www.ubuntu.com - sistema operacional

1 1

Reading the new XML files into R
by epidata-list＠lists.umanitoba.ca 12 Jun '11

12 Jun '11

To save me maybe re-inventing the wheel, has anyone looked at reading the new native XML files directly into R, ideally with SQL queries. There is an XML package for R http://www.omegahat.org/RSXML/c but it would be really useful if someone more capable than me, had already put together a method. Many thanks, Graham

1 20

release candidate of Manager
by epidata-list＠lists.umanitoba.ca 10 Jun '11

10 Jun '11

Dear All A "feature complete" version of EpiData Manager has been uploaded today - the expectation (read as "hope") is, that this version with bug removal can be released as version 1.0 * *NEW: *Create reports in HTML format using "Tools" * Default settings for a project * Change font and colour in program settings The reports on project structure (question lists, value label lists, combined lists) are produced when you click on tools menu. All epx/epz files in that folder are included. (Notice they must be of most recent format). For Entry Client a similar version will be released within the next days . The following aspecs will be fixed/clarified before release: a. A Proper About box will be added. b. For reports a grid of included files will be shown to the user - just like "add structure" to allow for in- and exclusion of single files. c. known and reported bugs. d. any bugs reported from now on by users. e. updating of documentation on the wiki. Another interesting development has taken place in the last week. We have localised an open-source based cross-platform testing system for interactive simulation of user behavior. We will spend some energy and time to develop validation tests to secure that the visual interface behaves properly after each new compilation. More news on this later. Therefore please: Now is the time to challenge the software and report problems and viewpoints on the products - before releasing as v1. Documentation and instruction files could be more elaborate obviously - this will come later. In practice it has turned out, that unless features work almost without instruction, users will not use it. Kind regards Torsten Christiansen Jens Lauritsen EpiData Association Denmark

1 5

new version of command line tool
by epidata-list＠lists.umanitoba.ca 06 Jun '11

06 Jun '11

Yesterday a new version of the command line template parser tool was compiled based on the most recent html structure for the epx files. Users can read more about this tool on: http://www.epidata.org/dokuwiki/doku.php/documentation:templateformat (the documentation will be finalised, but in general is correct) And get the updated version from: http://www.epidata.dk/testing.php regards Jens Lauritsen EpiData Association

1 1

request of help
by epidata-list＠lists.umanitoba.ca 03 Jun '11

03 Jun '11

Dear members in the group, I will be teaching an course of bioestatistics to a group of psychiatry residents. The idea is to teach statistic methodology but using scientific papers related to psychiatry. I would appreciate if any of you have some papers with the databases that could provide me. Best wishes Jose Farfan Mexico

1 0

new test release of Manager and Entry Client
by epidata-list＠lists.umanitoba.ca 02 Jun '11

02 Jun '11

Dear List. Today we have release updated versions of the EpiData Manager and EntryClient programs. The release today is a minor test release with focus on Manager enhancements and user customization. Common fixed/changes: * Is is now possible to change font type, size and colour in both programs. This is done independently of the project. * A long standing problem where a projectfile contained exactly 13 fields would make the programs crash during load is finally fixed. * Other minor bugs related to importing/exporting to stata have been fixed. Manager: * The "import/add structure" window now gives the user the option of keeping/renaming field and valuelabels if types and names match (produces smaller/smarter project files if there are overlapping fields/valuelabels). * In the program settings is it possible to define a default setup for a new project. * Using the "tools" no longer requires the project file to be open. * Using "importing structure" could under some circumstances produce fields without fieldnames. * Several bugfixes related to deleting field(s) involved in calculations, jumps, etc. Entry: * Font changes as described above. * Some minor bug-fixes. During the most recent weeks experimentation has been made with the creation of pdf formatted reports on structure, labels etc. Unfortunately we had to stop the pdf development due to various reasons and time constraints. Instead an html based solution will be created. This will be part of next test release. Kind regards, Jens Lauritsen and Torsten Bonde Christiansen EpiData Association.

1 3

fill fields from COMMENT LEGAL file
by epidata-list＠lists.umanitoba.ca 01 Jun '11

01 Jun '11

I'm studying the copydata example to learn how to extract several variables from a file identified in a COMMENT LEGAL statement. In A3.chk, the statement "TYPE COMMENT atransfer" appears. What is atransfer? I don't see where it has been defined as a variable and I don't find it as a system variable. Further, the program seems to be parsing the string in atransfer according to the position of punctuation. Where is the format of this string defined? Perhaps to be a bit more explicit: I have file with a list of CHWs: District, Village, CHW name and a survey conducted by the CHWs. As the data is entered for the survey, I'd like to present pick lists to facilitate entry. The data is quite strictly hierarchical: Villages occur within Districts, an CHWs live in villages. Thank you,

1 1

RE EpiData-list Digest, Vol 91, Issue 9 - EpiData statistics documentation.
by epidata-list＠lists.umanitoba.ca 30 May '11

30 May '11

Lu 30 mai 2011 Hello Jamie, Thank's for your quick answer. You are the spirit of EpiData users. I'll try to get this reference book. I would be glad to work with you about this topic and for the cause of EpiData. If it may help Jens, it will help us all. Cordialement, Réjean Dion, M.D. epidata-list-request@lists.umanitoba.ca@lists.umanitoba.ca Envoyé par : epidata-list-bounces(a)lists.umanitoba.ca 2011-05-30 13:00 Veuillez répondre à epidata-list(a)lists.umanitoba.ca A epidata-list(a)lists.umanitoba.ca cc Objet EpiData-list Digest, Vol 91, Issue 9 Send EpiData-list mailing list submissions to epidata-list(a)lists.umanitoba.ca To subscribe or unsubscribe via the World Wide Web, visit http://lists.umanitoba.ca/mailman/listinfo/epidata-list or, via email, send a message with subject or body 'help' to epidata-list-request(a)lists.umanitoba.ca You can reach the person managing the list at epidata-list-owner(a)lists.umanitoba.ca When replying, please edit your Subject line so it is more specific than "Re: Contents of EpiData-list digest..." EpiData-list mailing list ___________________________________ Today's Topics: 1. EpiData statistics documentation. (epidata-list(a)lists.umanitoba.ca) 2. Re: EpiData statistics documentation. (epidata-list(a)lists.umanitoba.ca) ---------------------------------------------------------------------- Message: 1 Date: Mon, 30 May 2011 10:30:02 -0400 From: epidata-list(a)lists.umanitoba.ca Subject: [EpiData-list] EpiData statistics documentation. To: epidata-list(a)lists.umanitoba.ca Message-ID: <OFCEE0BD3B.825D84B0-ON852578A0.004A567F-852578A0.004FA7C8(a)isn.rtss.qc.ca> Content-Type: text/plain; charset="ISO-8859-1" Lu 30 mai 2011 Hello EpiData users, I'm a new subscriber to this list. I used recently EpiData Analysis for a matched case-control with a controls:case ratio of 38:19 or 2:1 (TAB DISEASE EXPOSURE COMMONID /O /t /ex). In the results output, and in the online help, for the crude table, I could'nt find the type of test used, for example, which kind of Chi-square (I was able to deduced it was uncorrected), of odds ratio (maximun likelihoodestimate [MLE]?) and associated confidence interval (95%?), and Fishers exact test (bilateral?). In the adjusted table, the OR [MLE?] is provided with a CI (MLE and 95%?), followed by a Chi-square (unspecified kind) and another p value (which kind?). The provided p values are rounded at 3 digits after the decimal point. The last column provide p-exact values (Fishers?), and the one on the adjusted line seems to refer to another Chi-squared provided at the bottom of the table. Finally, the exact p value for the adjusted results (e.g.: 0.3140) doesn't match with the "probability of MLE >= value of MLE of OR (e.g. 6.10) if population OR = 1.0" that I obtain (e.g..: 0.00196) with the command MATCH COMMONID DISEASE EXPOSURE of Epi Info 6.04d DOS (which is probably equivalent to an exact test), and seem discordant with the values of the CI (e.g.: 1.62 to 34.17). Using the cohort approach (TAB DISEASE EXPOSURE /RR /t /ex), I found similar lack of documentation about statistical tests. I would suggest the following to the conceptors of EpiData: - provide some documentation about statistical tests used in Analysis (do any of the users have those answers?); - provide notes at the bottom of tables to eliminate any ambiguities about statistical methods; - provide p values in the ajusted table with 4 digits after the decimal point. This software work pretty well, but need some ajustments. Keep on the good work. Best regards, Réjean Réjean Dion, M.D. Médecin-conseil en santé publique (maladies infectieuses) Institut national de santé publique du Québec (INSPQ) / Laboratoire de santé publique du Québec (LSPQ) 20045, chemin Sainte-Marie, Sainte-Anne-de-Bellevue (Québec) Canada H9X 3R5 Tél.: (514) 457-2070 poste 325 Fax.: (514) 457-6346 Courriel / Lotus Notes: rejean.dion(a)inspq.qc.ca ------------------------------ Message: 2 Date: Mon, 30 May 2011 11:02:37 -0400 From: epidata-list(a)lists.umanitoba.ca Subject: Re: [EpiData-list] EpiData statistics documentation. To: epidata-list(a)lists.umanitoba.ca Message-ID: <BLU0-SMTP98AFF72F724FC7BE103867FD7B0(a)phx.gbl> Content-Type: text/plain; charset="iso-8859-1" Hi Réjean, The current version uses the statistics recommended in the book Statistics with Confidence: Confidence intervals and statistical guidelines, 2nd Edition Douglas Altman (Editor), David Machin (Editor), Trevor Bryant(Editor), Stephen Gardner (Editor) ISBN: 978-0-7279-1375-3 The help file references this book only with respect to Confidence Intervals. I agree that, in time, a more complete description of the statistical results is warranted. That's something we might want to work together on, to give Jens a hand--what do you think? This version of Analysis does not provide an option to increase the number of decimal positions shown with statistics, but more precise results are available from results variables such as $exact The command result shows all of the results accessible to the user. Jamie Hockin ------------------------------ End of EpiData-list Digest, Vol 91, Issue 9 *******************************************

1 0