[Feature] Import WakaTime data by uploading JSON file

[solonovamax]

Currently, WakaTime has a feature that allows you to export your heartbeat data by downloading a large JSON file.

It would be nice if you could upload this file to wakapi instead of only supporting imports from wakatime itself. For example, if there was another, hypothetical, service that also tracked your programming just as wakatime/wakapi does, and it exported in a wakapi compatible format, it would be really nice if this could be imported.

Thus, it would remove a dependence on wakatime itself and further generalize the concept.

Yes, it would allow people to forge their times for the leaderboards, but it could be optionally disabled on the public instance (or would just fail if the times are suspicious/in the top 1%)

[muety]

I like it. There could be "just" another importer besides the WakaTime one, e.g. a JsonFileUploadImport or a CsvFileUploadImport.

Right now, we don't have a leaderboard yet, so we can postpone these thoughts to later. But probably, I'd offer an option to exclude heartbeats from certain import sources from the leaderboard then.

[Sternbach-Software]

Yeah, this is a barrier of entry for me. A deal breaker: I can't move to Wakapi if I can't take my old time with me.

[muety]

What is the structure and format of your old data?

[Sternbach-Software]

@muety Here are my Kotlin data classes modeling the JSON exported by WakaTime (heartbeats):

data class HeartbeatsJSON( //this is the high level representation of the JSON
    val user: User,
    val range: Range,
    val days: List<HeartbeatDay>,
)

data class User(
        val bio: String? = null,
        val city: String? = null,
        val color_scheme: String? = "Dark",
        val created_at: String? = "2021-09-10T14:12:04Z",
        val date_format: String? = "YYYY-MM-DD",
        val default_dashboard_range: String? = "Last 7 Days",
        val display_name: String? = "Anonymous User",
        val durations_slice_by: String? = "Language",
        val email: String? = "software@gmail.com",
        val full_name: String? = null,
        val has_premium_features: Boolean? = false,
        val human_readable_website: String? = null,
        val id: String? = "uuid",
        val is_email_confirmed: Boolean? = true,
        val is_email_public: Boolean? = false,
        val is_hireable: Boolean? = false,
        val is_onboarding_finished: Boolean? = true,
        val languages_used_public: Boolean? = false,
        val last_heartbeat_at: String? = "2022-01-09T01:09:17Z",
        val last_plugin: String? = "wakatime/v1.32.1 (darwin-21.1.0-arm64) go1.17.5 AndroidStudio/Arctic Fox | 2020.3.1 Patch 4",
        val last_plugin_name: String? = "Android Studio",
        val last_project: String? = "Project1",
        val location: String? = null,
        val logged_time_public: Boolean? = false,
        val modified_at: String? = null,
        val needs_payment_method: Boolean? = false,
        val photo: String? = "https://wakatime.com/photo/aaa",
        val photo_public: Boolean? = true,
        val plan: String? = "basic",
        val profile_url: String? = "https://wakatime.com/@id",
        val profile_url_escaped: String? = "https://wakatime.com/@id",
        val public_email: String? = null,
        val public_profile_time_range: String? = "last_7_days",
        val share_all_time_badge: String? = null,
        val show_machine_name_ip: Boolean? = false,
        val time_format_24hr: String? = null,
        val time_format_display: String? = "text",
        val timeout: Int? = 15,
        val timezone: String? = "America/New_York",
        val username: String? = null,
        val website: String? = null,
        val weekday_start: Int? = 0,
        val writes_only: Boolean? = false
    )

data class Range(
        val end: Long,
        val start: Long
    )

data class HeartbeatDay(
    val date: String = "2021-09-10",
    val heartbeats: List<Heartbeat>,
)
data class Heartbeat(
    val branch: String = "master",
    val category: String = "coding",
    val created_at: String = "2021-09-10T14:14:44Z",
    val cursorpos: Int? = null,
    val dependencies: List<Any> = listOf(),
    val entity: String = "C:/Users/user/projects/project/src/code.kt",
    val id: String = "UUID",
    val is_write: Boolean? = null,
    val language: String = "Kotlin",
    val lineno: Int? = null,
    val lines: Int = 367,
    val machine_name_id: String = "UUID",
    val project: String = "Project",
    val time: Double = 1631283258.132,
    val type: String = "file",
    val user_agent_id: String = "UUID",
    val user_id: String = "UUID"
)

[Sternbach-Software]

WakaTime also has an option to export summaries. Here are the classes for that (note that in the previous list, a question mark means that the variable can be null; in this list, I was not as careful to mark what can be null or not):

data class SummaryJSON(
    val user: User,//same as in HeartbeatsJSON
    val range: Range, //^^^^
    val days: List<SummaryDay>
)
data class SummaryDay(
        val categories: List<Category>,
        val date: String,
        val dependencies: List<Category>,
        val editors: List<Category>,
        val grand_total: GrandTotal,
        val languages: List<Category>,
        val machines: List<Machine>,
        val operating_systems: List<Category>,
        val projects: List<Project>
    )
data class Category(
        val decimal: String? = "2.55",
        val digital: String? = "2:33:04",
        val hours: Int? = 2,
        val minutes: Int? = 33,
        val name: String? = "Coding",
        val percent: Double? = 100.0,
        val seconds: Int? = 4,
        val text: String? = "2 hrs 33 mins",
        val total_seconds: Double? = 9184.140000
    )
data class GrandTotal(
        val decimal: String? = "2.55",
        val digital: String? = "2:33:04",
        val hours: Int? = 2,
        val minutes: Int? = 33,
        val text: String? = "2 hrs 33 mins",
        val total_seconds: Double? = 9184.140000
    )
data class Machine(
        val decimal: String? = "2.55",
        val digital: String? = "2:33:04",
        val hours: Int? = 2,
        val minutes: Int? = 33,
        val machine_name_id: String? = "UUID",
        val name: String? = "Coding",
        val percent: Double? = 100.0,
        val seconds: Int? = 4,
        val text: String? = "2 hrs 33 mins",
        val total_seconds: Double? = 9184.140000
    )
data class Project(
        val branches: List<Category>,
        val categories: List<Category>,
        val dependencies: List<Category>,
        val editors: List<Category>,
        val entites: List<Entity>,
        val grand_total: GrandTotal,
        val languages: List<Category>,
        val machines: List<Machine>,
        val name: String?,
        val operating_systems: List<Category>,
    )
data class Entity(
        val decimal: String? = "2.55",
        val digital: String? = "2:33:04",
        val hours: Int? = 2,
        val minutes: Int? = 33,
        val name: String? = "Coding",
        val percent: Double? = 100.0,
        val seconds: Int? = 4,
        val text: String? = "2 hrs 33 mins",
        val total_seconds: Double? = 9184.140000,
        val type: String?
    )

[muety]

You can use Wakapi's WakaTime import feature. Go to Settings -> Integrations -> WakaTime.

[IgnisDa]

@muety I would like to work on this feature. Would you accept a PR?

[muety]

Hi @IgnisDa, thanks for your willingness to help on this. I'd want to wait for #453 to be implemented first, though, because I think these two features go well together. I'll keep you posted.

[IgnisDa]

I'll keep you posted.

Thanks, that'd be great!

[tazz4843]

You can use Wakapi's WakaTime import feature. Go to Settings -> Integrations -> WakaTime.

This throws a 402 Payment Required outside of the last two weeks of activity, which is not desirable. Is there any progress on this feature?

[muety]

Is there any progress on this feature?

Not yet, but I'll bump priority to A. I'm planning to dedicate a whole one or two days to Wakapi somewhere in the next week to push out a batch of bug fixes and new features. Perhaps this will be included.

[muety]

Update: Won't implement JSON import, but instead directly integrate with WakaTime's data dump API, see #323.