meta data is different between ImageJ and RBioformats

[gvogler]

Hi!

I am getting different results for CXD files imported with ImageJ vs RBioformats. A couple of fields (related to image scale) are all set to '1' in RBioformats, whereas the same file metadata in ImageJ is correctly identified (using Bioformats plugin, 5.8.1).

Any ideas? This is a great R library, and for an automated file analysis I really would need this feature. A sample file can be found here:

https://www.dropbox.com/s/5txk8b1rorzs7m8/Data1051.cxd?dl=0

Best,

Geo

[aoles]

Dear Geo,

many thanks for reporting this. Thank you for sharing the sample file. It would be really helpful if you could also provide some code which illustrates the issue, along with the expected values. For example, you could use the following template to filter by pattern.

Cheers,
Andrzej

meta = read.metadata("Data1051.cxd")
globalMetadata(meta, pattern=...)

[gvogler]

Sure thing - here is the code I am using:

library(RBioFormats)
library(reshape2)

meta.data <- read.metadata("Data1051.cxd")

  # Get timecodes for all frames
  g <- melt(meta.data$globalMetadata)
  g$L1 <- paste0(g$L1,"    ")
  g$Field <- as.numeric(unlist(lapply(g$L1, function(x) substring(x,gregexpr(' ',x)[[1]][1]+1,gregexpr(' ',x)[[1]][2]-1))))
  g$Type <- unlist(lapply(g$L1, function(x) substring(x,gregexpr(' ',x)[[1]][[2]],gregexpr(' ',x)[[1]][[3]])))
  g$Type2 <- unlist(lapply(g$Type, function(x)gsub("[[:space:]]", "", x)))
  
  cxd_fields_timecodes <- g[which(g$Type2 == "Time_From_Start"),]
  cxd_fields_timecodes <- cxd_fields_timecodes[order(cxd_fields_timecodes$Field),]
  cxd_fields_intervals <- g[which(g$Type2 == "Time_From_Last"),]
  cxd_fields_intervals <- cxd_fields_intervals[order(cxd_fields_intervals$Field),]
  
  scale_factor <- g$value[which(g$L1 == "factor    ")]
  magnification <- g$value[which(g$L1 == "magnification    ")]
  # Add time interval to Metadata
  meta.data$coreMetadata$time_interval <- mean(cxd_fields_intervals$value, na.rm = TRUE)
  meta.data$coreMetadata$resolution <-   scale_factor * magnification

Scale factor and magnification should be (from ImageJ and the original recording software)

factor = 0.260000	;Current Calibration Scale Factor
magnification = 2.000000	;Current Calibration Magnification

but is

1

[aoles]

Thank you for providing the sample code! Actually, there might be a problem with the postprocessing, as the annotation itself seems to be extracted properly:

library(magrittr)

res = meta.data %>% globalMetadata(pattern="factor|magnification")

## $factor
## [1] "0.260000\t;Current Calibration Scale Factor"
##
## $magnification
## [1] "2.000000\t;Current Calibration Magnification"

You could then use something along the following lines to perform the conversion to numeric.

lapply(res, function(x) as.numeric(gsub("\t;.*$", "", x)))

## $factor
## [1] 0.26
## 
## $magnification
## [1] 2

[gvogler]

Perfect solution! I'm glad the problem was outside Rbioformats :-) Excellent package!

[gvogler]

Hi, Andrzej! I’m running into an issue with extracting meta data (again), this time it is too slow from within R. I modified my code to use showinf from the bftools instead, but I was wondering if there is a hidden parameter in read.metadata to speed this up. For now, I’m using the following code:   meta.data <- as.data.frame(system(paste0("./showinf -nopix ", filelist$cxd[f]), intern = TRUE)) With read.metadata it seems it still reads the entire file before extracting the meta information. Do you have any ideas? Many thanks, Geo From: "Andrzej K. Oleś" <notifications@github.com> Reply-To: aoles/RBioFormats <reply@reply.github.com> Date: Wednesday, March 21, 2018 at 7:22 AM To: aoles/RBioFormats <RBioFormats@noreply.github.com> Cc: "geo.vogler@gmail.com" <geo.vogler@gmail.com>, Author <author@noreply.github.com> Subject: Re: [aoles/RBioFormats] meta data is different between ImageJ and RBioformats (#11) Thank you for providing the sample code! Actually, there might be a problem with the postprocessing, as the annotation itself seems to be extracted properly: library(magrittr) meta.data%>% globalMetadata(pattern="factor|magnification") ## $factor ## [1] "0.260000\t;Current Calibration Scale Factor" ## ## $magnification ## [1] "2.000000\t;Current Calibration Magnification" — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[aoles]

Hi Geo,

thank you for reaching out and for reporting the disappointing performance of read.metadata. I will look int this, I've opened a dedicated issue in order to better keep track of it.

Best,
Andrzej

[gvogler]

Hi, Andrzej! I am stuck while trying to install Rbioformats in Windows 10. Rtools and devtools are installed, packages can be compile from source. rJava compiled fine and loads properly. However, when running I get

biocLite("aoles/RBioFormats")

I get the following output: BioC_mirror: https://bioconductor.org Using Bioconductor 3.7 (BiocInstaller 1.30.0), R 3.5.1 (2018-07-02). Installing github package(s) ‘aoles/RBioFormats’ Downloading GitHub repo aoles/RBioFormats@master Installing package into ‘C:/Users/Geo/Documents/R/win-library/3.5’ (as ‘lib’ is unspecified) * installing *source* package 'RBioFormats' ... ** R ** inst ** byte-compile and prepare package for lazy loading ** help *** installing help indices converting help for package 'RBioFormats' finding HTML links ... done AnnotatedImage-class html finding level-2 HTML links ... done AnnotatedImageList-class html BioFormats.version html FormatTools html ImageMetadata-class html RBioFormats html checkJavaMemory html dimorder html metadata html metadataAccessors html mockFile html read.image html read.metadata html read.omexml html seriesCount html write.image html ** building package indices ** installing vignettes ** testing if installed package can be loaded trying URL 'https://downloads.openmicroscopy.org/bio-formats/5.9.1/artifacts/bioformats_package.jar' Content type 'application/x-java-archive' length 31779356 bytes (30.3 MB) ================================================== downloaded 30.3 MB Error: package or namespace load failed for 'RBioFormats': .onLoad failed in loadNamespace() for 'RBioFormats', details: call: .jfield("loci/formats/FormatTools", "S", "VERSION") error: cannot find class loci/formats/FormatTools Error: loading failed Execution halted ERROR: loading failed * removing 'C:/Users/Geo/Documents/R/win-library/3.5/RBioFormats' In R CMD INSTALL Warning message: In i.p(...) : installation of package ‘C:/Users/Geo/AppData/Local/Temp/RtmpU7bsbM/remotes13fc5a4431eb/aoles-RBioFormats-9d104f6’ had non-zero exit status Do you have an idea what might go wrong? Many thanks in advance, Geo From: "Andrzej K. Oleś" <notifications@github.com> Reply-To: aoles/RBioFormats <reply@reply.github.com> Date: Sunday, May 13, 2018 at 1:03 PM To: aoles/RBioFormats <RBioFormats@noreply.github.com> Cc: Geo Vogler <GEO.VOGLER@GMAIL.COM>, State change <state_change@noreply.github.com> Subject: Re: [aoles/RBioFormats] meta data is different between ImageJ and RBioformats (#11) Hi Geo, thank you for reaching out and for reporting the disappointing performance of read.metadata. I will look int this, I've opened a dedicated issue<#12> in order to better keep track of it. Best, Andrzej — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub<#11 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AaGUenMsW-xwAM96YwOFxAUFg_aRCrKjks5tyJF3gaJpZM4Sppuu>.

[aoles]

Long time no see :)

Thanks for reporting this, I will have a look into it. I have opened a dedicated issue #14 to keep track of it, if you have any further comments or observations please post them there.

Cheers,
Andrzej