Version 1.12.0
Major improvements in SFM in version 1.12.0:
- Deprecated web harvester. This will be replaced by other approaches in future releases that involve sending URLs to external web archives. Existing web harvests will not be deleted, but no new web harvests will be performed.
- Deprecated ELK. This is replaced by TweetSets, which provides a more scalable approach for indexing social media posts in ElasticSearch. An existing ELK instance can continue to run, but no new social media posts will be loaded.
- To improve citability of datasets, added public links field to collections and citation guidance to documentation.
- Added automatic, configurable seed deletion for seeds that have been suspended, deleted, protected, etc.
- Added support for deactivating credentials, for credentials which are no longer valid.
- Removed pinning of transitive dependencies to assist with managing dependency change.
- Worked to enable clean shutdown (status code 0) of containers.
- Switched to used Twarc's Json2Csv for exporting tweets.
Changes in sfm-docker:
- Upgraded processing containers to newer Ubuntu and added / upgraded tools.
- Removed ELK and web harvester.
Changes in docs:
- Fixed links to Twitter docs.
- Add citation guidance page.
- Updated processing container docs to reflect changes / additions.
- Corrected smoke test instructions.
- Deprecated web harvester and ELK.
- Updated Twitter data dictionary to reflect change in Twitter export.
- Update Export documentation to add detail about time zones.
Known issues:
- No significant known issues
Release notes for specific components:
- sfm-ui
- sfm-utils
- sfm-twitter-harvester
- sfm-flickr-harvester
- sfm-weibo-harvester
- sfm-tumblr-harvester
For a complete list of tickets, see sfm-ui milestone 1.12.0.
To upgrade to this version of SFM, follow the general upgrade instructions. In your .env
file, remove the WEB HARVESTER CONFIGURATION SECTION and the WEB_REQS line.
Also, change the versions of twitterrestexporter and twitterstreamexporter to 1.12.1 in your docker-compose.yml
file.
After SFM is upgraded, execute docker system prune -a
.