August 2019 - EpiData-list - University of Manitoba Mailing Lists

Re: [EpiData-list] Missing data values when exporting to Stata (EpiData development and support)
by EpiData development and support 28 Aug '19

28 Aug '19

Hi, Exporting the .dta file from Analysis throws the same error in Python as the Stata 14-file from EpiData Manager. Unsure if this is a Python error or Stata-export error. I have used all versions earlier in R so probably something to do with Python Pandas read_stata-function. Kind Regards Henning

1 1

Re: [EpiData-list] Missing data values when exporting to Stata
by EpiData development and support 27 Aug '19

27 Aug '19

Hi, again I tried exporting to Stata v13 and v14 from EpiData Manager (same version av before), but Python throws an error when reading both files. Unsure what the errors are. I can’t find where to export in EpiData Analysis v1.4.0.0. I tried “File -> Save” as .dta, but the file is not a Stata-file. I tried looking in the documentation for an export command, but can’t find one. What am I missing? Kind Regards Henning

1 1

Re: [EpiData-list] Missing data values when exporting to Stata
by EpiData development and support 24 Aug '19

24 Aug '19

I will download the Analysis app and try that. I will also try to export to Stata 14 as the Stata version does not matter with regards to Python. Thanks Kind Regards Henning Langen Stokmo Consultant (Nuclear Medicine) // Department of Nuclear Medicine, Oslo University Hospital Radiology Resident // Department of Medical Imaging, Vestre Viken HF, Drammen Hospital PhD-student // Department of Nuclear Medicine, Oslo University Hospital | Oslo University Functional and Molecular Imaging Research Group (https://www.ous-research.no/revheim/) ResearchGate: https://www.researchgate.net/profile/Henning_Langen_Stokmo Sent from Canary (https://canarymail.io/) > On Friday, Aug 23, 2019 at 7:00 PM, <epidata-list-request(a)lists.umanitoba.ca (mailto:epidata-list-request@lists.umanitoba.ca)> wrote: > Send EpiData-list mailing list submissions to > epidata-list(a)lists.umanitoba.ca > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.umanitoba.ca/mailman/listinfo/epidata-list > or, via email, send a message with subject or body 'help' to > epidata-list-request(a)lists.umanitoba.ca > > You can reach the person managing the list at > epidata-list-owner(a)lists.umanitoba.ca > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of EpiData-list digest..." > > > EpiData-list mailing list > ___________________________________ > > Today's Topics: > > 1. Re: Missing data values when exporting to Stata > (EpiData development and support) > 2. Re: Missing data values when exporting to Stata > (EpiData development and support) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 22 Aug 2019 23:40:02 +0200 > From: EpiData development and support > <epidata-list(a)lists.umanitoba.ca> > To: epidata-list(a)lists.umanitoba.ca > Subject: Re: [EpiData-list] Missing data values when exporting to > Stata > Message-ID: <041A2CAB-A14A-4F26-8389-3410D8489A76(a)gmail.com> > Content-Type: text/plain; charset=utf-8 > > Sorry for not giving any information about the versions and system I use. > > I use EpiData Manager v4.4.2.1 on a MacBook Pro 13? (2018) running Mojave 10.14.6 to export. The exported files are to Stata v12. > The .dta files are opened in Python v3.6.8 via PyCharm IDE CE 2019.2 and imported to a Panda dataframe with the command ?pd.read_stata(<filename>). The empty lines in the Stata-file gets correctly set as ?NaN? in Pandas, but the value label for the missing category is set to ?100.0? instead of ?NaN? as I expect it should be after coding this in EpiData. > > Kind Regards > Henning L. Stokmo > Phd-Student, Oslo University Hospital > > > > On 22 Aug 2019, at 19:00, epidata-list-request(a)lists.umanitoba.ca wrote: > > > > Send EpiData-list mailing list submissions to > > epidata-list(a)lists.umanitoba.ca > > > > To subscribe or unsubscribe via the World Wide Web, visit > > http://lists.umanitoba.ca/mailman/listinfo/epidata-list > > or, via email, send a message with subject or body 'help' to > > epidata-list-request(a)lists.umanitoba.ca > > > > You can reach the person managing the list at > > epidata-list-owner(a)lists.umanitoba.ca > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of EpiData-list digest..." > > > > > > EpiData-list mailing list > > ___________________________________ > > > > Today's Topics: > > > > 1. Re: Missing data values when exporting to Stata > > (EpiData development and support) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Wed, 21 Aug 2019 21:43:41 +0200 > > From: EpiData development and support > > <epidata-list(a)lists.umanitoba.ca> > > To: epidata-list(a)lists.umanitoba.ca > > Subject: Re: [EpiData-list] Missing data values when exporting to > > Stata > > Message-ID: <CE3439F1-72F7-4F8A-90F1-313DF876BD64(a)epidata.dk> > > Content-Type: text/plain; charset=utf-8 > > > > As I remember the Stata value should be > > .a > > But please specify which version of Stata and if you are using Manager export or the new analysis, which may directly save the file in Stata format. > > > > Best wishes > > Jens Lauritsen > > EpiData Association > > Denmark > > > > Den 21. august 2019 14.04.31 CEST, EpiData development and support <epidata-list(a)lists.umanitoba.ca> skrev: > > > Hi, > > > > > > I have some missing data that is encoded as ?9? in the database and in > > > the value label list it is flagged as missing. When exporting to CSV > > > the value is obviously set as ?9? as expected. > > > > > > When I export it as Stata I assume this value ?9? gets encoded to > > > Stata? definition of a missing value, but when I read the data in > > > Python with the ?pd.read_stata?-function it defines the category value > > > ?9? as 100.0 and not as missing data (NaN) like the blanks in EpiData. > > > > > > Eksample > > > One column has 80 values. 19 of these are encoded as ?9? (missing), 58 > > > of these are blank, and 3 are coded as ?1? (Yes) in EpiData. After > > > exporting to Stata and reading in Python I get > > > > > > 19 values in category ?100? > > > 3 values in category ?Yes? > > > 58 values NaN (i.e. missing) > > > > > > Why aren?t the ?9? category correctly coded as missing when exporting > > > to Stata, or am I missing something here? > > > > > > Kind Regards > > > Henning > > > _______________________________________________ > > > EpiData-list mailing list > > > EpiData-list(a)lists.umanitoba.ca > > > http://lists.umanitoba.ca/mailman/listinfo/epidata-list > > > > -- > > Sent from my Android device with K-9 Mail. Please excuse my brevity. > > > > ------------------------------ > > > > ________________________________________ > > EpiData-list(a)lists.umanitoba.ca > > http://lists.umanitoba.ca/mailman/listinfo/epidata-list > > > > > > End of EpiData-list Digest, Vol 183, Issue 2 > > ******************************************** > > > > ------------------------------ > > Message: 2 > Date: Fri, 23 Aug 2019 11:21:07 +0200 > From: EpiData development and support > <epidata-list(a)lists.umanitoba.ca> > To: epidata-list(a)lists.umanitoba.ca > Subject: Re: [EpiData-list] Missing data values when exporting to > Stata > Message-ID: <C7C018D0-96BA-45C8-8AFF-7F9FC92D31AF(a)epidata.dk> > Content-Type: text/plain; charset=utf-8 > > You spotted an error in export to Stata 12. > > If you export to Stata 14 it works as desired. > > A easier way is to use the new Analysis for export, Analysis has options for choice of Stata version a d works on a Mac. > > Can the python use Stata 14? > > Best wishes > Jens Lauritsen > EpiData Association > Denmark > > > > > Den 22. august 2019 23.40.02 CEST, EpiData development and support <epidata-list(a)lists.umanitoba.ca> skrev: > > Sorry for not giving any information about the versions and system I > > use. > > > > I use EpiData Manager v4.4.2.1 on a MacBook Pro 13? (2018) running > > Mojave 10.14.6 to export. The exported files are to Stata v12. > > The .dta files are opened in Python v3.6.8 via PyCharm IDE CE 2019.2 > > and imported to a Panda dataframe with the command > > ?pd.read_stata(<filename>). The empty lines in the Stata-file gets > > correctly set as ?NaN? in Pandas, but the value label for the missing > > category is set to ?100.0? instead of ?NaN? as I expect it should be > > after coding this in EpiData. > > > > Kind Regards > > Henning L. Stokmo > > Phd-Student, Oslo University Hospital > > > > > > > On 22 Aug 2019, at 19:00, epidata-list-request(a)lists.umanitoba.ca > > wrote: > > > > > > Send EpiData-list mailing list submissions to > > > epidata-list(a)lists.umanitoba.ca > > > > > > To subscribe or unsubscribe via the World Wide Web, visit > > > http://lists.umanitoba.ca/mailman/listinfo/epidata-list > > > or, via email, send a message with subject or body 'help' to > > > epidata-list-request(a)lists.umanitoba.ca > > > > > > You can reach the person managing the list at > > > epidata-list-owner(a)lists.umanitoba.ca > > > > > > When replying, please edit your Subject line so it is more specific > > > than "Re: Contents of EpiData-list digest..." > > > > > > > > > EpiData-list mailing list > > > ___________________________________ > > > > > > Today's Topics: > > > > > > 1. Re: Missing data values when exporting to Stata > > > (EpiData development and support) > > > > > > > > > > > ---------------------------------------------------------------------- > > > > > > Message: 1 > > > Date: Wed, 21 Aug 2019 21:43:41 +0200 > > > From: EpiData development and support > > > <epidata-list(a)lists.umanitoba.ca> > > > To: epidata-list(a)lists.umanitoba.ca > > > Subject: Re: [EpiData-list] Missing data values when exporting to > > > Stata > > > Message-ID: <CE3439F1-72F7-4F8A-90F1-313DF876BD64(a)epidata.dk> > > > Content-Type: text/plain; charset=utf-8 > > > > > > As I remember the Stata value should be > > > .a > > > But please specify which version of Stata and if you are using > > Manager export or the new analysis, which may directly save the file in > > Stata format. > > > > > > Best wishes > > > Jens Lauritsen > > > EpiData Association > > > Denmark > > > > > > Den 21. august 2019 14.04.31 CEST, EpiData development and support > > <epidata-list(a)lists.umanitoba.ca> skrev: > > > > Hi, > > > > > > > > I have some missing data that is encoded as ?9? in the database and > > in > > > > the value label list it is flagged as missing. When exporting to CSV > > > > the value is obviously set as ?9? as expected. > > > > > > > > When I export it as Stata I assume this value ?9? gets encoded to > > > > Stata? definition of a missing value, but when I read the data in > > > > Python with the ?pd.read_stata?-function it defines the category > > value > > > > ?9? as 100.0 and not as missing data (NaN) like the blanks in > > EpiData. > > > > > > > > Eksample > > > > One column has 80 values. 19 of these are encoded as ?9? (missing), > > 58 > > > > of these are blank, and 3 are coded as ?1? (Yes) in EpiData. After > > > > exporting to Stata and reading in Python I get > > > > > > > > 19 values in category ?100? > > > > 3 values in category ?Yes? > > > > 58 values NaN (i.e. missing) > > > > > > > > Why aren?t the ?9? category correctly coded as missing when > > exporting > > > > to Stata, or am I missing something here? > > > > > > > > Kind Regards > > > > Henning > > > > _______________________________________________ > > > > EpiData-list mailing list > > > > EpiData-list(a)lists.umanitoba.ca > > > > http://lists.umanitoba.ca/mailman/listinfo/epidata-list > > > > > > -- > > > Sent from my Android device with K-9 Mail. Please excuse my brevity. > > > > > > ------------------------------ > > > > > > ________________________________________ > > > EpiData-list(a)lists.umanitoba.ca > > > http://lists.umanitoba.ca/mailman/listinfo/epidata-list > > > > > > > > > End of EpiData-list Digest, Vol 183, Issue 2 > > > ******************************************** > > > > _______________________________________________ > > EpiData-list mailing list > > EpiData-list(a)lists.umanitoba.ca > > http://lists.umanitoba.ca/mailman/listinfo/epidata-list > > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. > > ------------------------------ > > ________________________________________ > EpiData-list(a)lists.umanitoba.ca > http://lists.umanitoba.ca/mailman/listinfo/epidata-list > > > End of EpiData-list Digest, Vol 183, Issue 3 > ********************************************

1 0

Re: [EpiData-list] Missing data values when exporting to Stata
by EpiData development and support 23 Aug '19

23 Aug '19

Sorry for not giving any information about the versions and system I use. I use EpiData Manager v4.4.2.1 on a MacBook Pro 13” (2018) running Mojave 10.14.6 to export. The exported files are to Stata v12. The .dta files are opened in Python v3.6.8 via PyCharm IDE CE 2019.2 and imported to a Panda dataframe with the command “pd.read_stata(<filename>). The empty lines in the Stata-file gets correctly set as “NaN” in Pandas, but the value label for the missing category is set to “100.0” instead of “NaN” as I expect it should be after coding this in EpiData. Kind Regards Henning L. Stokmo Phd-Student, Oslo University Hospital > On 22 Aug 2019, at 19:00, epidata-list-request(a)lists.umanitoba.ca wrote: > > Send EpiData-list mailing list submissions to > epidata-list(a)lists.umanitoba.ca > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.umanitoba.ca/mailman/listinfo/epidata-list > or, via email, send a message with subject or body 'help' to > epidata-list-request(a)lists.umanitoba.ca > > You can reach the person managing the list at > epidata-list-owner(a)lists.umanitoba.ca > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of EpiData-list digest..." > > > EpiData-list mailing list > ___________________________________ > > Today's Topics: > > 1. Re: Missing data values when exporting to Stata > (EpiData development and support) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 21 Aug 2019 21:43:41 +0200 > From: EpiData development and support > <epidata-list(a)lists.umanitoba.ca> > To: epidata-list(a)lists.umanitoba.ca > Subject: Re: [EpiData-list] Missing data values when exporting to > Stata > Message-ID: <CE3439F1-72F7-4F8A-90F1-313DF876BD64(a)epidata.dk> > Content-Type: text/plain; charset=utf-8 > > As I remember the Stata value should be > .a > But please specify which version of Stata and if you are using Manager export or the new analysis, which may directly save the file in Stata format. > > Best wishes > Jens Lauritsen > EpiData Association > Denmark > > Den 21. august 2019 14.04.31 CEST, EpiData development and support <epidata-list(a)lists.umanitoba.ca> skrev: >> Hi, >> >> I have some missing data that is encoded as ?9? in the database and in >> the value label list it is flagged as missing. When exporting to CSV >> the value is obviously set as ?9? as expected. >> >> When I export it as Stata I assume this value ?9? gets encoded to >> Stata? definition of a missing value, but when I read the data in >> Python with the ?pd.read_stata?-function it defines the category value >> ?9? as 100.0 and not as missing data (NaN) like the blanks in EpiData. >> >> Eksample >> One column has 80 values. 19 of these are encoded as ?9? (missing), 58 >> of these are blank, and 3 are coded as ?1? (Yes) in EpiData. After >> exporting to Stata and reading in Python I get >> >> 19 values in category ?100? >> 3 values in category ?Yes? >> 58 values NaN (i.e. missing) >> >> Why aren?t the ?9? category correctly coded as missing when exporting >> to Stata, or am I missing something here? >> >> Kind Regards >> Henning >> _______________________________________________ >> EpiData-list mailing list >> EpiData-list(a)lists.umanitoba.ca >> http://lists.umanitoba.ca/mailman/listinfo/epidata-list > > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. > > ------------------------------ > > ________________________________________ > EpiData-list(a)lists.umanitoba.ca > http://lists.umanitoba.ca/mailman/listinfo/epidata-list > > > End of EpiData-list Digest, Vol 183, Issue 2 > ********************************************

1 1

Re: [EpiData-list] Missing data values when exporting to Stata
by EpiData development and support 22 Aug '19

22 Aug '19

Hi, I have some missing data that is encoded as ‘9’ in the database and in the value label list it is flagged as missing. When exporting to CSV the value is obviously set as ‘9’ as expected. When I export it as Stata I assume this value ‘9’ gets encoded to Stata’ definition of a missing value, but when I read the data in Python with the ‘pd.read_stata’-function it defines the category value ‘9’ as 100.0 and not as missing data (NaN) like the blanks in EpiData. Eksample One column has 80 values. 19 of these are encoded as ‘9’ (missing), 58 of these are blank, and 3 are coded as ‘1’ (Yes) in EpiData. After exporting to Stata and reading in Python I get 19 values in category ‘100’ 3 values in category ‘Yes’ 58 values NaN (i.e. missing) Why aren’t the ‘9’ category correctly coded as missing when exporting to Stata, or am I missing something here? Kind Regards Henning

1 1