- EpiData-list - University of Manitoba Mailing Lists

READ statement / Invalid dates / How to handle dates in analysis
by epidata-list＠lists.umanitoba.ca 12 Mar '07

12 Mar '07

READ statement / Invalid dates / How to handle dates in analysis 1. I submitted my previous email concerning the READ statement because: - on reading a particular file, a list of some 20+ invalid dates was produced - followed by a message that there were x valid records and 10 records excluded from analysis. I assumed that the 10 records were excluded because of the invalid dates. Subsequently, I have discovered that there was no connection between the invalid dates and the excluded records! In fact, the invalid records were records which had been deleted. SUGGESTION: When records are excluded because they are deleted records, could the message use the words "Deleted records" rather than "Excluded records". Just to avoid any confusion! 2. To respond to Jamie's email (Invalid dates), some of the invalid dates had been entered by EpiInfo, but about half had been entered using EpiData. EpiData Entry has no problems accepting a date such as 12/03/206. (I have also tried 12/03/0206 which just immediately reverts to 12/03/206). Yet EpiData Analysis sees such a date as invalid. SUGGESTION: EpiData Entry and Analysis should be consistent with their interpretation of invalid dates. 3. To respond to Jens' email (How to handle dates in analysis), illegal dates are not the issue I am addressing. Illegal dates are not accepted by EpiData and will therefore not be an issue in Analysis. I am addressing legal dates which may have been entered incorrectly, e.g. 12/03/2006 inadvertently entered as 12/03/206. EpiData accepts this date. But Analysis does not accept it as a valid date. 4. Once the dataset with invalid dates has been read in Analysis, it is not possible to carry out any commands which might include such dates. BROWSE, LIST, SAVEDATA for example will not operate (unless one specifies variables excluding those which have invalid values). SUGGESTION: Would it not be possible for Analysis to assign a standard valid value (e.g. 01/01/1900) to any invalid dates it comes across during the read process. A message to this effect could be added to the list of invalid dates (with their variable names and record numbers). It is then up to the user to edit the invalid dates. But then at least Analysis will be able to carry out all commands. Many thanks, Annemieke van Middelkoop Pretoria, South Africa

1 0

How to handle dates in analysis
by epidata-list＠lists.umanitoba.ca 11 Mar '07

11 Mar '07

Clarification: Neither EpiData Entry nor EpiData Analysis can have date fields with illegal dates. E.g. February 29th is only allowed in leap years. Epi6 could save illegal dates and never controlled the data while reading for correct content. >From the beginning of EpiData all date fields were included as "true dates", meaning that there cannot be a date value which does not exist in the past or future. (dates before year 1600 can give problems) For EpiData Analysis this has given some users problems since it was difficult to see in which records the date problems were located if some dates were rejected. Also some old dates before year 1800 have not been accomodated fully in analysis, but this should be a minor problem, since we never or very rarely have data from that period. And if we do one could split the dates into day month and year parts as three variables, although this would remove the automatic validation of the content of the data. I suggest that we look at the possibility of implementing: > c. That, if neither a. nor b. is an option, more detail about the > 'invalid' dates (record number, variable name) is provided so that > they can be edited. since we cannot use an illegal date for anything. Analysis with incorrect dates makes no sense. Once we find a workable principle for this the same technical principle and list can include also names of variables/fields, which will be in conflict with internal functions, e.g. year, date or tab. -- Jens Lauritsen Coordinator and initiator of EpiData Project http://www.epidata.dk

1 0

Learning material in Portuguese
by epidata-list＠lists.umanitoba.ca 10 Mar '07

10 Mar '07

I added a link to the learning material in Portuguese on the Download page (www.epidata.dk/download.php) - thanks to the group in Rio Janeiro. It is good if users start experimenting with development of "WIKI" based notes and materials. You can do so by at the address www.epidata.org/wiki After some experimentation period and establishment of a responsible group for the Wiki a separation can be made btw. "official" materials and materials at the responsibility of users. regards Jens Lauritsen EpiData Association

1 0

INVALID dates
by epidata-list＠lists.umanitoba.ca 10 Mar '07

10 Mar '07

Dear Friend, Epi Analysis is a sensitive tool to detect the invalid dates that can be corrected. Of course these invalid dated can be prevented altogether by inserting date ranges in the check files also before data entry. Epi Analysis also detects the corrupted records in the file while READing it, which EPI6 was not able to do. Hence screening of dataset while loading is an important for data cleaning also and should be retained. I fully endorse the option c. suggested by you. Just like providing record numbers of corrupted records, the record with invalid dates should be listed with record number and if possible the date variable name also. With regards, Dr. Shavinder Singh

1 0

Re: [EpiData-list] invalid dates
by epidata-list＠lists.umanitoba.ca 10 Mar '07

10 Mar '07

While a solution that flags dates as invalid/missing will be useful, another strategy for dealing with the invalid dates is to import them as text and use the built-in functions to convert valid ones to dates. There is still a potential problem with finding the invalid date records, but you have an opportunity to do this on your own terms. I assume that the dates were imported somehow into a .rec file, as entry would not allow such dates. I don't recall whether import checks date validity, but it does other field types. The invalid dates can readily be checked using the validation capacity of EpiData Entry and .chk files. This may be the best way to do this, as a detailed report of errors is produced. I have run into this type of error importing EpiInfo files and reading .dbf files, but not working with .rec files that I had control over. Jamie > > I have several concerns about this approach. > 1. Most important, records with 'invalid' dates are excluded from > analysis (10 in this example). The fact that the 'invalid' dates are > likely to be incorrect is no reason to not be able to analyse the > records. > 2. READing a large dataset with many dates can be a very slow process. > 3. The output given by Analysis (as shown above) is not very useful. > Neither record numbers nor variable names are identified. > > I would like to suggest: > a. That dates are not checked at all during the READ process. The > analyst can look for incorrect dates in Analysis, relevant to the > content of the data. > OR > b. That the user can specify whether to check the dates or not on > READing a dataset. > OR > c. That, if neither a. nor b. is an option, more detail about the > 'invalid' dates (record number, variable name) is provided so that > they can be edited. > > Annemieke van Middelkoop > Pretoria, South Africa >

1 0

Learning material
by epidata-list＠lists.umanitoba.ca 09 Mar '07

09 Mar '07

Hello all, Sometime a go, I tried to find out some learning/teaching material for EpiData entry in portuguese. Unfortunately, I didnt find any. Since this year is the first year that I'll teach a class about computer program for clinical research to pos-graduation students, I decided to make a small material in portuguese. A sort of introductory step by step guide for beginners and some little more complex examples. If any would like to take a look, it is available at : http://www.ipec.fiocruz.br/materialdidatico/Aplicativos_pesquisa/2007/EpiDa… By -- Abraço forte e que a força esteja com você, Pedro Emmanuel

1 0

RE: EpiData-list Digest, Vol 41, Issue 2
by epidata-list＠lists.umanitoba.ca 09 Mar '07

09 Mar '07

I agree with the approach of Annenmieke van Middelkoop about 'invalid' records after "Read". Bernard Branger, Nantes, France __________________________________________________________________________ I have several concerns about this approach. 1. Most important, records with 'invalid' dates are excluded from analysis (10 in this example). The fact that the 'invalid' dates are likely to be incorrect is no reason to not be able to analyse the records. 2. READing a large dataset with many dates can be a very slow process. 3. The output given by Analysis (as shown above) is not very useful. Neither record numbers nor variable names are identified. I would like to suggest: a. That dates are not checked at all during the READ process. The analyst can look for incorrect dates in Analysis, relevant to the content of the data. OR b. That the user can specify whether to check the dates or not on READing a dataset. OR c. That, if neither a. nor b. is an option, more detail about the 'invalid' dates (record number, variable name) is provided so that they can be edited. Annemieke van Middelkoop Pretoria, South Africa ------------------------------ ________________________________________ EpiData-list(a)lists.umanitoba.ca http://lists.umanitoba.ca/mailman/listinfo/epidata-list End of EpiData-list Digest, Vol 41, Issue 2 *******************************************

1 0

READ statement
by epidata-list＠lists.umanitoba.ca 07 Mar '07

07 Mar '07

When READing a dataset in Analysis, the program checks all dates in the dataset for 'valid' dates. For example, on reading a dataset DATA.REC, the following message is displayed: . Read Loading data C:\TEST\DATA.REC, please wait.. invalid year 1000, month 1, day 17 Error field DLOPV record: 1401 invalid year 5001, month 6, day 18 invalid year 200, month 1, day 16 invalid year 199, month 8, day 8 invalid year 200, month 2, day 2 invalid year 2200, month 12, day 3 invalid year 202, month 12, day 10 invalid year 202, month 12, day 10 invalid year 200, month 12, day 4 invalid year 200, month 12, day 4 invalid year 202, month 12, day 19 invalid year 603, month 5, day 24 invalid year 603, month 6, day 24 invalid year 199, month 1, day 15 invalid year 203, month 8, day 23 invalid year 203, month 8, day 23 invalid year 993, month 1, day 1 invalid year 199, month 9, day 19 invalid year 802, month 1, day 17 invalid year 199, month 4, day 18 invalid year 996, month 7, day 17 invalid year 206, month 7, day 31 invalid year 200, month 5, day 11 Errors reading data File name : C:\TEST\DATA.REC Fields: 222 Total records: 10460 Valid records: 10450 Excluded records: 10 I have several concerns about this approach. 1. Most important, records with 'invalid' dates are excluded from analysis (10 in this example). The fact that the 'invalid' dates are likely to be incorrect is no reason to not be able to analyse the records. 2. READing a large dataset with many dates can be a very slow process. 3. The output given by Analysis (as shown above) is not very useful. Neither record numbers nor variable names are identified. I would like to suggest: a. That dates are not checked at all during the READ process. The analyst can look for incorrect dates in Analysis, relevant to the content of the data. OR b. That the user can specify whether to check the dates or not on READing a dataset. OR c. That, if neither a. nor b. is an option, more detail about the 'invalid' dates (record number, variable name) is provided so that they can be edited. Annemieke van Middelkoop Pretoria, South Africa

1 0

Re: [EpiData-list] Entering time and date for outbreaks
by epidata-list＠lists.umanitoba.ca 28 Feb '07

28 Feb '07

Hi Charity Thanks very much for your formula. Its sounds really good. I had a go at putting it in a check file. Following through your example below 'Diatime' is my 'vtime2' and 'diadate' is my vdate2. I thought I would start off slowly by seeing if I coudl calculate Vhours2. I created a new field called Vhours2 in my rec file and then I put the following code into the check file. DIATIME AFTER ENTRY LET VHOURS2=INT(DIATIME)+(FRAC(DIATIME)/0.6)+(DIADATE*24) END END Epi accepted the code but nothing appears in the vhours2 field when I enter information in the diadate and diatime fields. Can you see what I am doing wrong? thanks Christine Roseveare Analyst Regional Public Health 570 9194 027 495 9671 >>> <epidata-list(a)lists.umanitoba.ca> 26/02/2007 7:24 p.m. >>> Hello Chrsitine, I prefer to enter times as a decimal field, i.e. ##.##. I can then put checks in place to make sure the minutes are in correct time format e.g. IF Frac(vtime) > 0.59 THEN .... Based on this decimal 24 hour time format, I have a formula which you can use to calculate the difference in hours between two time and date combinations. Assume that your first time and date are vtime1 and vdate1, and your second time and date are vtime2 and vdate2: First calculate the number of hours (since 31st December 1899) for both time and date combinations (store them as vhours1 and vhours2) using the following formulae: vhours2=Int(vtime2) + (Frac(vtime2)/0.6) + (vdate2*24) vhours1=Int(vtime1) + (Frac(vtime1)/0.6) + (vdate1*24) And then you can just calculate the time difference /incubation period into a different variable! timediff=vhours2-vhours1 You could also have done something similar and perhaps simpler using EpiData's Time2Num function which will convert a decimal time into a number between 0 and 1 representing a fraction of a day - I just prefer to work with hours. To explain the hours calculation formula for you (only need to read below if you care how the formula works): Int(vtime) takes the integer part of the time, and Frac(vtime) takes the fractional part of the time. These are separated so Frac(vtime) alone can be converted from minutes into fraction of an hour (e.g. from 0.30 to 0.50). The date integer (representing the number of days since 31 Dec 1899) is then multiplied by 24 to get the number of hours since 31 Dec 1899. This 1899 date is relevant only because it is how EpiData stores the date as an integer - we do not have to worry about this because we only care about the difference between the two dates. Hope this helps! Charity Charity Jenkins Data Processing Officer Centre for Behavioural Research in Cancer The Cancer Council Victoria http://www.cancervic.org.au/cbrc<https://webmail.accv.org.au/exchweb/bin/redir.asp?URL=http://www.cancervic.…> On 2/26/07, epidata-list(a)lists.umanitoba.ca <epidata-list(a)lists.umanitoba.ca> wrote: > > Hello everyone > > I am after advice about the best way to record time and date > information when collecting information about food poisoning outbreaks, > given epidata does not have a time/date field. We want this information > to estimate incubation periods, in order to get clues about the likely > illness. > > Incubation > Incubation = time from when someone is exposed to the time they become > unwell. > For example someone may eat a buffet meal at 7pm on 21 February and > become unwell at 1am on 22 February = incubation of 6 hours. > > Current method > At the moment we: > a) enter information in two fields. OnsetDay is recorded in a date > field, and onsettime in a numeric field (####) > b) list the two fields (using epi info) to view the information > c) calculate incubation manually. > > Is there a better/easier way to do this? Also what is the recommended > way for entering 24 clock time - should 7.30pm be 1930 or 19.30 > > thanks very much > > Christine > > > > Christine Roseveare > Analyst > Regional Public Health > 570 9194 > 027 495 9671 > > > > > -- > The information contained in this email and > any attachments is confidential and may be > legally privileged. If you have received this > message in error, please notify the sender > immediately and remove all copies of the message, > including any attachments. Any views or > opinions expressed in this email (unless > otherwise stated) may not represent those of > Hutt Valley DHB. > Thank you. > ** Disclaimer added by HVDHB ** > > _______________________________________________ > EpiData-list mailing list > EpiData-list(a)lists.umanitoba.ca > http://lists.umanitoba.ca/mailman/listinfo/epidata-list > -- The information contained in this email and any attachments is confidential and may be legally privileged. If you have received this message in error, please notify the sender immediately and remove all copies of the message, including any attachments. Any views or opinions expressed in this email (unless otherwise stated) may not represent those of Hutt Valley DHB. Thank you. ** Disclaimer added by HVDHB **

1 4

work-arounds for data entry on more than one computer
by epidata-list＠lists.umanitoba.ca 28 Feb '07

28 Feb '07

Epi-Data users: What strategies have you used to get around the limitation of only being able to input the data on one computer? We are thinking of splitting the file into two - with Type A respondents in one file/on one computer and Type B respondents in another file/on a different computer and then combining the resulting datasets in Stata. Will this work? Are there other/better strategies? Thanks, Kristine Hopkins khopkins(a)prc.utexas.edu

1 3