Copy API for DataSets, Attributes and Groups

[apdofficial]

Hello everyone,

First of all, thanks for the fantastic job with HighFive!

I did not find any HighFive copy API. I have a file A with a path /group-1/group-2/.../group-n, and in group-n, there is a dataset. I want to copy the entire path from file A into some other file B in some group, including the groups, group attributes, dataset, and dataset attributes. Is there anything that HighFive provides to support this?

I wrote a small code that can partially do that (see below). In my case, I also use offset and count as I select a region of interest from the data. If you find it useful, I can create a PR and adjust it/ improve as needed.

/**
 * @brief Copy an attribute to a HighFive::AnnotateTrait.
 *
 * @param attribute to be added
 * @param dst [out] supporting HighFive::AnnotateTrait into which the attribute is added
 */
template <typename Derivate>
void CopyAttribute(HighFive::Attribute const& attribute, HighFive::AnnotateTraits<Derivate>& dst) {
  auto const dtype = attribute.getDataType();
  auto target = dst.createAttribute(attribute.getName(), attribute.getSpace(), dtype);
  if (dtype.isVariableStr()) {
    target.write(attribute.template read<std::string>());
  } else if (dtype.getClass() == HighFive::DataTypeClass::VarLen) {
    throw std::runtime_error("Support for variable length sequences is not implemented");
  } else {
    std::vector<char> buffer(dtype.getSize());
    attribute.read(buffer.data(), dtype);
    target.write_raw(buffer.data(), dtype);
  }
}

/**
 * @brief Add all supported attributes to the dst group from the src group.
 *
 * @param src from which the attributes are read
 * @param dst [out] into which the attributes are added
 */
template <typename Derivate>
void CopyAttributes(HighFive::AnnotateTraits<Derivate> const& src,
                    HighFive::AnnotateTraits<Derivate>& dst) {
  for (auto const& attribute_name : src.listAttributeNames()) {
    CopyAttribute(src.getAttribute(attribute_name), dst);
  }
}

/**
 * @brief Copy <src> dataset into <dst> group
 *
 * @param src dataset to be copied
 * @param dst [out] HighFive::Group into which the dataset is copied
 * @param offset beginning of the selection to be copied
 * @param count length of the selection to be copied
 */
void CopyDataset(HighFive::DataSet const& src,
                 HighFive::Group& dst,
                 std::vector<size_t> const& offset,
                 std::vector<size_t> const& count) {
  auto const name = src.getPath().substr(path.find_last_of("/\\") + 1);
  auto const selection = src.select(offset, count);
  auto const space = selection.getMemSpace();
  auto const dtype = selection.getDataType();
  auto new_dataset = dst.createDataSet(name, space, dtype);
  std::vector<char> buffer(space.getElementCount() * dtype.getSize());
  selection.read(buffer.data(), dtype);
  new_dataset.write_raw(buffer.data(), dtype);
  CopyAttributes(src, new_dataset);
}

/**
 * @brief Copy groups and their attributes.
 *
 * @param groups should represent a path form root to the bottom.
 * @param dst root below which to copy the groups.
 * @return HighFive::Group bottom of the path
 */
HighFive::Group CopyGroups(std::vector<HighFive::Group> const& groups, HighFive::Group dst) {
  auto group = groups.begin();
  if (group == groups.end()) {
    return dst;
  }
  CopyAttributes(*group, dst);
  while (++group != groups.end()) {
    auto const name = group->getPath().substr(path.find_last_of("/\\") + 1);
    dst = dst.createGroup(name);
    CopyAttributes(*group, dst);
  }
  return dst;
}

Thank you for your support in advance.

Regards,
Andrej

[1uc]

Thank you, that seems like useful functionality. @alkino found that HDF5 has a function H5Ocopy. I'd like to experiment with that to see if it would solves the issue with less work on our side.

[apdofficial]

Great! Let me know when you have any updates or if I can help. Indeed, ' H5Ocopy` seems to be the perfect tool for this.