Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pageserver: trace slow requests when compact_l0 #10505

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

pageserver: trace slow requests when compact_l0 #10505

wants to merge 1 commit into from
+60 −1

Conversation

skyzh
Copy link
Member

@skyzh skyzh commented Jan 24, 2025

for incident investigations

@skyzh @problame
pageserver: trace slow requests when compact_l0
9b081a8 
Signed-off-by: Alex Chi Z <chi@neon.tech>
Co-authored-by: Christian Schwarz <christian@neon.tech>
@skyzh skyzh force-pushed the skyzh/compact-l0-slow-trace branch from 7e6dddc to 9b081a8 Compare January 24, 2025 22:33
@github-actions GitHub Actions
Copy link

7403 tests run: 6430 passed, 592 failed, 381 skipped (full report)

Failures on Postgres 17

# Run all failed tests locally:
scripts/pytest -vv -n $(nproc) -k "test_auth_failures[debug-pg17-True] or test_pageserver_multiple_keys[debug-pg17] or test_auth_failures[debug-pg17-False] or test_pageserver_auth[debug-pg17] or test_pageserver_key_reload[debug-pg17] or test_compute_auth_to_pageserver[debug-pg17] or test_ancestor_branch[debug-pg17] or test_empty_config[debug-pg17-None] or test_null_config[debug-pg17] or test_empty_config[debug-pg17-application/json] or test_null_body[debug-pg17] or test_fully_custom_config[debug-pg17] or test_branch_behind[debug-pg17] or test_branching_with_pgbench[debug-pg17-flat-1-10] or test_branching_unnormalized_start_lsn[debug-pg17] or test_branching_with_pgbench[debug-pg17-cascade-1-10] or test_non_uploaded_branch_is_deleted_after_restart[debug-pg17] or test_cannot_branch_from_non_uploaded_branch[debug-pg17] or test_cannot_create_endpoint_on_non_uploaded_timeline[debug-pg17] or test_branching_while_stuck_find_gc_cutoffs[debug-pg17] or test_non_uploaded_root_timeline_is_deleted_after_restart[debug-pg17] or test_duplicate_creation[debug-pg17] or test_compute_pageserver_connection_stress[debug-pg17] or test_branch_creation_before_gc[debug-pg17] or test_basebackup_error[debug-pg17] or test_create_multiple_timelines_parallel[debug-pg17] or test_timeline_init_break_before_checkpoint_recreate[debug-pg17-return] or test_local_corruption[debug-pg17] or test_timeline_init_break_before_checkpoint[debug-pg17] or test_timeline_create_break_after_dir_creation[debug-pg17] or test_timeline_init_break_before_checkpoint_recreate[debug-pg17-exit] or test_clog_truncate[debug-pg17] or test_build_info_metric[debug-pg17] or test_combocid_delete[debug-pg17] or test_combocid_multi_insert[debug-pg17] or test_combocid_update[debug-pg17] or test_combocid[debug-pg17] or test_combocid_lock[debug-pg17] or test_change_pageserver[debug-pg17] or test_lsof_pageserver_pid[debug-pg17] or test_pageserver_gc_compaction_idempotent[debug-pg17-before_restart] or test_image_layer_compression[debug-pg17-True] or test_pageserver_gc_compaction_idempotent[debug-pg17-after_restart] or test_sharding_compaction[debug-pg17-None-None-False] or test_uploads_and_deletions[debug-pg17-legacy] or test_pageserver_compaction_circuit_breaker[debug-pg17] or test_uploads_and_deletions[debug-pg17-tiered] or test_image_layer_compression[debug-pg17-False] or test_sharding_compaction[debug-pg17-4-32768-False] or test_sharding_compaction[debug-pg17-4-16-False] or test_sharding_compaction[debug-pg17-4-32768-True] or test_create_snapshot[debug-pg17] or test_backward_compatibility[debug-pg17] or test_versions_mismatch[debug-pg17-combination_nnnnn] or test_versions_mismatch[debug-pg17-combination_ononn] or test_versions_mismatch[debug-pg17-combination_onnnn] or test_versions_mismatch[debug-pg17-combination_nnnoo] or test_compute_catalog[debug-pg17] or test_compute_create_databases[debug-pg17] or test_dropdb_with_subscription[debug-pg17] or test_compute_drop_role[debug-pg17] or test_default_locales[debug-pg17] or test_sql_exporter_metrics_smoke[debug-pg17-autoscaling] or test_sql_exporter_metrics_e2e[debug-pg17-compute] or test_sql_exporter_metrics_e2e[debug-pg17-autoscaling] or test_perf_counters[debug-pg17] or test_sql_exporter_metrics_smoke[debug-pg17-compute] or test_compute_migrations_e2e[debug-pg17-3] or test_compute_migrations_e2e[debug-pg17-5] or test_compute_migrations_e2e[debug-pg17-4] or test_compute_migrations_retry[debug-pg17] or test_compute_migrations_e2e[debug-pg17-1] or test_compute_migrations_e2e[debug-pg17-2] or test_compute_migrations_e2e[debug-pg17-6] or test_compute_migrations_e2e[debug-pg17-8] or test_compute_migrations_e2e[debug-pg17-9] or test_compute_migrations_e2e[debug-pg17-7] or test_compute_migrations_e2e[debug-pg17-10] or test_compute_migrations_e2e[debug-pg17-11] or test_config[debug-pg17] or test_safekeepers_reconfigure_reorder[debug-pg17] or test_crafted_wal_end[debug-pg17-vanilla-simple] or test_crafted_wal_end[debug-pg17-vanilla-last_wal_record_xlog_switch] or test_crafted_wal_end[debug-pg17-vanilla-last_wal_record_xlog_switch_ends_on_page_boundary] or test_crafted_wal_end[debug-pg17-vanilla-last_wal_record_crossing_segment] or test_crafted_wal_end[debug-pg17-vanilla-wal_record_crossing_segment_followed_by_small_one] or test_crafted_wal_end[debug-pg17-interpreted-simple] or test_crafted_wal_end[debug-pg17-interpreted-last_wal_record_xlog_switch_ends_on_page_boundary] or test_crafted_wal_end[debug-pg17-interpreted-last_wal_record_xlog_switch] or test_crafted_wal_end[debug-pg17-interpreted-last_wal_record_crossing_segment] or test_crafted_wal_end[debug-pg17-interpreted-wal_record_crossing_segment_followed_by_small_one] or test_createdb[debug-pg17-file_copy] or test_createdb[debug-pg17-wal_log] or test_dropdb[debug-pg17] or test_ddl_forwarding_invalid_db[debug-pg17] or test_ddl_forwarding_role_specs[debug-pg17] or test_partial_evict_tenant[debug-pg17-relative_spare] or test_broken_tenants_are_skipped[debug-pg17] or test_min_resident_size_override_handling[debug-pg17-None] or test_pageserver_evicts_until_pressure_is_relieved[debug-pg17-relative_equal] or test_pageserver_respects_overridden_resident_size[debug-pg17-relative_equal] or test_pageserver_falls_back_to_global_lru[debug-pg17-relative_equal] or test_partial_evict_tenant[debug-pg17-relative_equal] or test_fast_growing_tenant[debug-pg17-relative_spare] or test_min_resident_size_override_handling[debug-pg17-400] or test_fast_growing_tenant[debug-pg17-relative_equal] or test_statvfs_pressure_usage[debug-pg17] or test_statvfs_error_handling[debug-pg17] or test_statvfs_pressure_min_avail_bytes[debug-pg17] or test_secondary_mode_eviction[debug-pg17] or test_createuser[debug-pg17] or test_endpoint_crash[debug-pg17-\U0001f4a3] or test_endpoint_crash[debug-pg17-trigger_panic] or test_endpoint_crash[debug-pg17-trigger_segfault] or test_fullbackup[debug-pg17] or test_gc_aggressive[debug-pg17] or test_gc_index_upload[debug-pg17] or test_hot_standby_gc[debug-pg17-True] or test_hot_standby[debug-pg17] or test_2_replicas_start[debug-pg17] or test_replica_query_race[debug-pg17] or test_hot_standby_gc[debug-pg17-False] or test_hot_standby_feedback[debug-pg17] or test_gin_redo[debug-pg17] or test_import_from_pageserver_small[debug-pg17] or test_import_from_vanilla[debug-pg17] or test_fsm_truncate[debug-pg17] or test_extensions[debug-pg17] or test_large_schema[debug-pg17] or test_gc_of_remote_layers[debug-pg17] or test_installed_extensions[debug-pg17] or test_issue_5878[debug-pg17-default_generation] or test_issue_5878[debug-pg17-same_generation] or test_layer_bloating[debug-pg17] or test_logging_event_count[debug-pg17-trace] or test_logging_event_count[debug-pg17-info] or test_logging_event_count[debug-pg17-error] or test_logging_event_count[debug-pg17-debug] or test_logging_event_count[debug-pg17-warn] or test_obsolete_slot_drop[debug-pg17] or test_large_records[debug-pg17] or test_replication_shutdown[debug-pg17] or test_slots_and_branching[debug-pg17] or test_subscriber_synchronous_commit[debug-pg17] or test_lr_with_slow_safekeeper[debug-pg17] or test_ondemand_wal_download_in_replication_slot_funcs[debug-pg17] or test_logical_replication[debug-pg17] or test_restart_endpoint[debug-pg17] or test_restart_endpoint[debug-pg17] or test_lsn_mapping[debug-pg17-True] or test_ts_of_lsn_api[debug-pg17] or test_lsn_mapping[debug-pg17-False] or test_get_lsn_by_timestamp_cancelled[debug-pg17] or test_multixact[debug-pg17] or test_cli_timeline_list[debug-pg17] or test_cli_tenant_create[debug-pg17] or test_cli_tenant_list[debug-pg17] or test_cli_start_stop[debug-pg17] or test_cli_ipv4_listeners[debug-pg17] or test_cli_start_stop_multi[debug-pg17] or test_nbtree_pagesplit_cycleid[debug-pg17] or test_neon_two_primary_endpoints_fail[debug-pg17] or test_neon_cli_basics[debug-pg17] or test_neon_extension_compatibility[debug-pg17] or test_neon_extension[debug-pg17] or test_neon_extension_auto_upgrade[debug-pg17] or test_next_xid[debug-pg17] or test_multixid_wraparound_import[debug-pg17] or test_import_at_2bil[debug-pg17] or test_normal_work[debug-pg17-3-1] or test_ondemand_download_large_rel[debug-pg17] or test_layer_download_cancelled_by_config_location[debug-pg17] or test_ondemand_download_timetravel[debug-pg17] or test_compaction_downloads_on_demand_without_image_creation[debug-pg17] or test_download_remote_layers_api[debug-pg17] or test_compaction_downloads_on_demand_with_image_creation[debug-pg17] or test_layer_download_timeouted[debug-pg17] or test_neon_superuser[debug-pg17] or test_oid_overflow[debug-pg17] or test_old_request_lsn[debug-pg17] or test_ondemand_download_replica[debug-pg17-None] or test_ondemand_download_pg_xact[debug-pg17-None] or test_ondemand_download_after_wal_switch[debug-pg17] or test_ondemand_download_pg_xact[debug-pg17-4] or test_ondemand_download_replica[debug-pg17-4] or test_slow_flush[debug-pg17-pageserver-stop] or test_slow_flush[debug-pg17-tenant-detach] or test_on_demand_wal_download[debug-pg17] or test_pageserver_http_get_wal_receiver_not_found[debug-pg17] or test_pageserver_http_get_wal_receiver_success[debug-pg17] or test_pageserver_http_api_client[debug-pg17] or test_pageserver_http_api_client_auth_enabled[debug-pg17] or test_deletion_queue_recovery[debug-pg17-validate-keep] or test_deletion_queue_recovery[debug-pg17-no-validate-lose] or test_emergency_mode[debug-pg17] or test_multi_attach[debug-pg17] or test_deferred_deletion[debug-pg17] or test_deletion_queue_recovery[debug-pg17-validate-lose] or test_deletion_queue_recovery[debug-pg17-no-validate-keep] or test_eviction_across_generations[debug-pg17] or test_old_index_time_threshold[debug-pg17] or test_upgrade_generationless_local_file_paths[debug-pg17] or test_generations_upgrade[debug-pg17] or test_local_only_layers_after_crash[debug-pg17] or test_pageserver_catchup_while_compute_down[debug-pg17] or test_pageserver_getpage_throttle[debug-pg17] or test_throttle_fair_config_is_settable_but_ignored_in_mgmt_api[debug-pg17] or test_throttle_fair_config_is_settable_but_ignored_in_config_toml[debug-pg17] or test_pageserver_small_inmemory_layers[debug-pg17-True] or test_pageserver_small_inmemory_layers[debug-pg17-False] or test_idle_checkpoints[debug-pg17] or test_pageserver_restart[debug-pg17] or test_pageserver_reconnect[debug-pg17] or test_pageserver_reconnect_failure[debug-pg17] or test_metric_collection[debug-pg17] or test_metric_collection_cleans_up_tempfile[debug-pg17] or test_location_conf_churn[debug-pg17-1] or test_secondary_downloads[debug-pg17] or test_secondary_background_downloads[debug-pg17] or test_live_migration[debug-pg17] or test_location_conf_churn[debug-pg17-2] or test_location_conf_churn[debug-pg17-3] or test_heatmap_uploads[debug-pg17] or test_pageserver_restarts_under_worload[debug-pg17] or test_pg_regress[debug-pg17-None] or test_isolation[debug-pg17-None] or test_sql_regress[debug-pg17-None] or test_sql_regress[debug-pg17-4] or test_pg_regress[debug-pg17-4] or test_isolation[debug-pg17-4] or test_cancellations[debug-pg17] or test_pg_waldump[debug-pg17] or test_physical_and_logical_replication_slot_not_copied[debug-pg17] or test_aux_not_logged_at_replica[debug-pg17] or test_physical_replication_config_mismatch_max_connections[debug-pg17] or test_physical_replication_config_mismatch_max_locks_per_transaction[debug-pg17] or test_physical_replication_config_mismatch_max_prepared[debug-pg17] or test_physical_replication_config_mismatch_too_many_known_xids[debug-pg17] or test_physical_replication[debug-pg17] or test_prefetch[debug-pg17-None] or test_prefetch[debug-pg17-4] or test_pitr_gc[debug-pg17] or test_readonly_node[debug-pg17] or test_timetravel[debug-pg17] or test_readonly_node_gc[debug-pg17] or test_read_validation[debug-pg17] or test_read_validation_neg[debug-pg17] or test_remote_storage_backup_and_restore[debug-pg17-True-local_fs] or test_remote_storage_backup_and_restore[debug-pg17-False-local_fs] or test_remote_storage_backup_and_restore[debug-pg17-True-mock_s3] or test_timeline_deletion_with_files_stuck_in_upload_queue[debug-pg17] or test_empty_branch_remote_storage_upload[debug-pg17] or test_remote_storage_backup_and_restore[debug-pg17-True-real_s3] or test_remote_storage_backup_and_restore[debug-pg17-False-mock_s3] or test_remote_storage_upload_queue_retries[debug-pg17] or test_remote_timeline_client_calls_started_metric[debug-pg17] or test_empty_branch_remote_storage_upload_on_restart[debug-pg17] or test_remote_storage_backup_and_restore[debug-pg17-False-real_s3] or test_pageserver_recovery[debug-pg17] or test_replica_start_wait_subxids_finish[debug-pg17] or test_replica_too_many_known_assigned_xids[debug-pg17] or test_replica_start_with_prepared_xacts_with_subxacts[debug-pg17] or test_replica_start_scan_clog[debug-pg17] or test_replica_start_with_prepared_xacts_with_many_subxacts[debug-pg17] or test_replica_start_at_running_xacts[debug-pg17] or test_replica_start_repro_visibility_bug[debug-pg17] or test_replica_start_with_prepared_xacts[debug-pg17-True] or test_replica_start_with_prepared_xacts[debug-pg17-False] or test_replica_start_with_too_many_unused_xids[debug-pg17] or test_replica_start_scan_clog_crashed_xids[debug-pg17] or test_role_grants[debug-pg17] or test_tenant_s3_restore[debug-pg17] or test_fixture_restart[debug-pg17] or test_sharding_split_compaction[debug-pg17-compact-shard-ancestors-persistent] or test_sharding_split_compaction[debug-pg17-compact-shard-ancestors-localonly] or test_sharding_split_compaction[debug-pg17-compact-shard-ancestors-enqueued] or test_sharding_split_stripe_size[debug-pg17-65536] or test_sharding_split_unsharded[debug-pg17] or test_sharding_split_compaction[debug-pg17-None] or test_sharding_split_failures[debug-pg17-failure3] or test_sharding_split_smoke[debug-pg17] or test_sharding_split_stripe_size[debug-pg17-None] or test_sharding_smoke[debug-pg17] or test_sharding_split_failures[debug-pg17-failure5] or test_sharding_split_failures[debug-pg17-failure6] or test_sharding_split_failures[debug-pg17-failure0] or test_sharding_split_failures[debug-pg17-failure2] or test_sharding_split_failures[debug-pg17-failure7] or test_sharding_split_failures[debug-pg17-failure8] or test_sharding_ingest_gaps[debug-pg17] or test_sharding_split_failures[debug-pg17-failure11] or test_sharding_split_failures[debug-pg17-failure4] or test_sharding_split_failures[debug-pg17-failure1] or test_sharding_split_offloading[debug-pg17] or test_sharding_split_failures[debug-pg17-failure9] or test_sharding_split_failures[debug-pg17-failure10] or test_sharding_split_failures[debug-pg17-failure13] or test_sharding_split_failures[debug-pg17-failure14] or test_sharding_split_failures[debug-pg17-failure15] or test_sharding_split_failures[debug-pg17-failure16] or test_sharding_split_failures[debug-pg17-failure17] or test_sharding_split_failures[debug-pg17-failure18] or test_sharding_split_failures[debug-pg17-failure19] or test_sharding_split_failures[debug-pg17-failure12] or test_top_tenants[debug-pg17] or test_sharding_split_failures[debug-pg17-failure20] or test_sharding_unlogged_relation[debug-pg17] or test_sharding_gc[debug-pg17] or test_sharding_backpressure[debug-pg17] or test_storage_controller_onboard_detached[debug-pg17] or test_storage_controller_smoke[debug-pg17-combination_nnnnn] or test_storage_controller_onboarding[debug-pg17-False] or test_storage_controller_smoke[debug-pg17-combination_ooonn] or test_storage_controller_passthrough[debug-pg17] or test_storage_controller_restart[debug-pg17] or test_storage_controller_smoke[debug-pg17-combination_ononn] or test_storage_controller_compute_hook[debug-pg17] or test_storage_controller_smoke[debug-pg17-combination_onnnn] or test_storage_controller_smoke[debug-pg17-combination_nnnoo] or test_storage_controller_onboarding[debug-pg17-True] or test_storage_controller_s3_time_travel_recovery[debug-pg17] or test_storage_controller_tenant_conf[debug-pg17] or test_node_status_after_restart[debug-pg17] or test_storage_controller_heartbeats[debug-pg17-failure0] or test_storage_controller_stuck_compute_hook[debug-pg17] or test_storage_controller_compute_hook_retry[debug-pg17] or test_storage_controller_compute_hook_revert[debug-pg17] or test_storage_controller_heartbeats[debug-pg17-failure1] or test_storage_controller_debug_apis[debug-pg17] or test_storage_controller_auth[debug-pg17] or test_storage_controller_tenant_deletion[debug-pg17] or test_storage_controller_heartbeats[debug-pg17-failure3] or test_storage_controller_re_attach[debug-pg17] or test_lock_time_tracing[debug-pg17] or test_tenant_import[debug-pg17-None-local_fs] or test_skip_drain_on_secondary_lag[debug-pg17] or test_storage_controller_shard_scheduling_policy[debug-pg17] or test_storcon_cli[debug-pg17] or test_tenant_import[debug-pg17-4-local_fs] or test_tenant_import[debug-pg17-None-real_s3] or test_tenant_import[debug-pg17-4-real_s3] or test_storage_controller_metadata_health[debug-pg17-None] or test_graceful_cluster_restart[debug-pg17] or test_storage_controller_node_deletion[debug-pg17-True] or test_storage_controller_step_down[debug-pg17] or test_background_operation_cancellation[debug-pg17] or test_storage_controller_timeline_crud_race[debug-pg17] or test_storage_controller_heartbeats[debug-pg17-failure2] or test_storage_controller_validate_during_migration[debug-pg17] or test_storage_controller_metadata_health[debug-pg17-2] or test_storage_controller_proxy_during_migration[debug-pg17-MigrationFailpoints.POST_DETACH] or test_storage_controller_ps_restarted_during_drain[debug-pg17] or test_storage_controller_node_deletion[debug-pg17-False] or test_storage_controller_proxy_during_migration[debug-pg17-MigrationFailpoints.POST_NOTIFY] or test_storage_controller_proxy_during_migration[debug-pg17-MigrationFailpoints.PRE_GENERATION_INC] or test_storage_controller_heartbeats[debug-pg17-failure4] or test_storage_controller_detach_lifecycle[debug-pg17] or test_storage_controller_leadership_transfer[debug-pg17-False] or test_timeline_delete_mid_live_migration[debug-pg17-MigrationFailpoints.PRE_AWAIT_LSN] or test_timeline_delete_mid_live_migration[debug-pg17-MigrationFailpoints.POST_NOTIFY] or test_storage_controller_detached_stopped[debug-pg17] or test_shard_preferred_azs[debug-pg17] or test_storage_controller_leadership_transfer[debug-pg17-True] or test_timeline_delete_mid_live_migration[debug-pg17-MigrationFailpoints.POST_DETACH] or test_timeline_delete_mid_live_migration[debug-pg17-MigrationFailpoints.PRE_GENERATION_INC] or test_multi_attached_timeline_creation[debug-pg17-MigrationFailpoints.POST_NOTIFY] or test_multi_attached_timeline_creation[debug-pg17-MigrationFailpoints.POST_DETACH] or test_multi_attached_timeline_creation[debug-pg17-MigrationFailpoints.PRE_GENERATION_INC] or test_storage_controller_node_flap_detach_race[debug-pg17] or test_scrubber_physical_gc[debug-pg17-None] or test_scrubber_tenant_snapshot[debug-pg17-None] or test_scrubber_physical_gc_ancestors[debug-pg17-None] or test_scrubber_scan_pageserver_metadata[debug-pg17-None] or test_scrubber_physical_gc_ancestors[debug-pg17-2] or test_scrubber_tenant_snapshot[debug-pg17-4] or test_scrubber_physical_gc[debug-pg17-4] or test_scrubber_scan_pageserver_metadata[debug-pg17-4] or test_scrubber_physical_gc_timeline_deletion[debug-pg17] or test_scrubber_physical_gc_ancestors_split[debug-pg17] or test_subscriber_branching[debug-pg17] or test_live_reconfig_get_evictions_low_residence_duration_metric_threshold[debug-pg17] or test_creating_tenant_conf_after_attach[debug-pg17] or test_tenant_config[debug-pg17] or test_tenant_config_patch[debug-pg17-storcon] or test_tenant_config_patch[debug-pg17-cplane] or test_tenant_delete_smoke[debug-pg17] or test_tenant_delete_races_timeline_creation[debug-pg17] or test_long_timeline_create_cancelled_by_tenant_delete[debug-pg17] or test_tenant_delete_stale_shards[debug-pg17] or test_tenant_delete_scrubber[debug-pg17] or test_subscriber_restart[debug-pg17] or test_subxacts[debug-pg17-vanilla] or test_subxacts[debug-pg17-interpreted] or test_tenant_reattach[debug-pg17-explicit] or test_tenant_reattach[debug-pg17-reset] or test_tenant_reattach[debug-pg17-reset_drop] or test_detach_while_attaching[debug-pg17] or test_detach_while_activating[debug-pg17] or test_metrics_while_ignoring_broken_tenant_and_reloading[debug-pg17] or test_tenant_detach_smoke[debug-pg17] or test_tenant_relocation[debug-pg17-with_load-minor] or test_emergency_relocate_with_branches_slow_replay[debug-pg17] or test_emergency_relocate_with_branches_createdb[debug-pg17] or test_tenant_relocation[debug-pg17-without_load-minor] or test_empty_tenant_size[debug-pg17] or test_branched_empty_timeline_size[debug-pg17] or test_branched_from_many_empty_parents_size[debug-pg17] or test_parent_within_horizon[debug-pg17] or test_only_heads_within_horizon[debug-pg17] or test_lsn_lease_size[debug-pg17-True] or test_lsn_lease_size[debug-pg17-False] or test_get_tenant_size_with_multiple_branches[debug-pg17] or test_branch_point_within_horizon[debug-pg17] or test_synthetic_size_while_deleting[debug-pg17] or test_tenant_tasks[debug-pg17] or test_timelines_parallel_endpoints[debug-pg17] or test_tenants_normal_work[debug-pg17] or test_tenant_creation_fails[debug-pg17] or test_pageserver_metrics_many_relations[debug-pg17] or test_pageserver_metrics_removed_after_detach[debug-pg17] or test_pageserver_with_empty_tenants[debug-pg17] or test_metrics_normal_work[debug-pg17] or test_create_churn_during_restart[debug-pg17] or test_tenants_many[debug-pg17] or test_tenants_attached_after_download[debug-pg17] or test_tenant_redownloads_truncated_file_on_startup[debug-pg17] or test_timeline_offload_persist[debug-pg17-False] or test_timeline_archive[debug-pg17-0] or test_timeline_offloading[debug-pg17-False] or test_timeline_retain_lsn[debug-pg17-offload-False] or test_timeline_offload_persist[debug-pg17-True] or test_timeline_offload_generations[debug-pg17] or test_timeline_offloading[debug-pg17-True] or test_timeline_retain_lsn[debug-pg17-offload-corrupt-True] or test_timeline_retain_lsn[debug-pg17-offload-no-restart-False] or test_timeline_retain_lsn[debug-pg17-offload-no-restart-True] or test_timeline_retain_lsn[debug-pg17-offload-parent-False] or test_timeline_retain_lsn[debug-pg17-offload-parent-True] or test_timeline_retain_lsn[debug-pg17-offload-True] or test_timeline_retain_lsn[debug-pg17-archive-True] or test_timeline_archive[debug-pg17-4] or test_timeline_retain_lsn[debug-pg17-None-False] or test_timeline_retain_lsn[debug-pg17-None-True] or test_timeline_offload_race_unarchive[debug-pg17-False] or test_timeline_retain_lsn[debug-pg17-offload-corrupt-False] or test_timeline_offload_race_unarchive[debug-pg17-True] or test_timeline_retain_lsn[debug-pg17-archive-False] or test_threshold_based_eviction[debug-pg17] or test_timeline_delete[debug-pg17] or test_delete_timeline_exercise_crash_safety_failpoints[debug-pg17-Check.RETRY_WITHOUT_RESTART-timeline-delete-before-index-deleted-at] or test_delete_timeline_exercise_crash_safety_failpoints[debug-pg17-Check.RETRY_WITHOUT_RESTART-timeline-delete-before-rm] or test_delete_timeline_exercise_crash_safety_failpoints[debug-pg17-Check.RETRY_WITHOUT_RESTART-timeline-delete-before-schedule] or test_delete_timeline_exercise_crash_safety_failpoints[debug-pg17-Check.RETRY_WITHOUT_RESTART-timeline-delete-after-rm] or test_delete_timeline_exercise_crash_safety_failpoints[debug-pg17-Check.RETRY_WITHOUT_RESTART-timeline-delete-before-index-delete] or test_delete_timeline_exercise_crash_safety_failpoints[debug-pg17-Check.RETRY_WITH_RESTART-timeline-delete-before-index-delete] or test_delete_timeline_exercise_crash_safety_failpoints[debug-pg17-Check.RETRY_WITHOUT_RESTART-timeline-delete-after-index-delete] or test_delete_timeline_exercise_crash_safety_failpoints[debug-pg17-Check.RETRY_WITH_RESTART-timeline-delete-before-index-deleted-at] or test_delete_timeline_exercise_crash_safety_failpoints[debug-pg17-Check.RETRY_WITH_RESTART-timeline-delete-before-schedule] or test_delete_timeline_exercise_crash_safety_failpoints[debug-pg17-Check.RETRY_WITH_RESTART-timeline-delete-before-rm] or test_delete_timeline_exercise_crash_safety_failpoints[debug-pg17-Check.RETRY_WITH_RESTART-timeline-delete-after-index-delete] or test_timeline_resurrection_on_attach[debug-pg17-True] or test_timeline_resurrection_on_attach[debug-pg17-False] or test_timeline_delete_fail_before_local_delete[debug-pg17] or test_concurrent_timeline_delete_stuck_on[debug-pg17-persist_deleted_index_part] or test_concurrent_timeline_delete_stuck_on[debug-pg17-in_progress_delete] or test_delete_timeline_client_hangup[debug-pg17] or test_timeline_delete_works_for_remote_smoke[debug-pg17] or test_delete_orphaned_objects[debug-pg17] or test_timeline_delete_resumed_on_attach[debug-pg17] or test_delete_timeline_exercise_crash_safety_failpoints[debug-pg17-Check.RETRY_WITH_RESTART-timeline-delete-after-rm] or test_ancestor_detach_branched_from[debug-pg17-True-True-at] or test_ancestor_detach_branched_from[debug-pg17-True-True-after] or test_ancestor_detach_branched_from[debug-pg17-True-True-head] or test_ancestor_detach_branched_from[debug-pg17-True-False-at] or test_ancestor_detach_branched_from[debug-pg17-True-False-earlier] or test_ancestor_detach_branched_from[debug-pg17-True-False-after] or test_ancestor_detach_branched_from[debug-pg17-True-False-head] or test_ancestor_detach_branched_from[debug-pg17-False-True-earlier] or test_ancestor_detach_branched_from[debug-pg17-False-True-at] or test_ancestor_detach_branched_from[debug-pg17-False-True-after] or test_ancestor_detach_branched_from[debug-pg17-False-True-head] or test_ancestor_detach_branched_from[debug-pg17-True-True-earlier] or test_ancestor_detach_branched_from[debug-pg17-False-False-at] or test_ancestor_detach_branched_from[debug-pg17-False-False-after] or test_ancestor_detach_branched_from[debug-pg17-False-False-head] or test_ancestor_detach_reparents_earlier[debug-pg17] or test_detached_receives_flushes_while_being_detached[debug-pg17] or test_compaction_induced_by_detaches_in_history[debug-pg17] or test_timeline_ancestor_detach_idempotent_success[debug-pg17-shards_initial_after0] or test_timeline_ancestor_detach_idempotent_success[debug-pg17-shards_initial_after1] or test_timeline_ancestor_detach_idempotent_success[debug-pg17-shards_initial_after2] or test_timeline_ancestor_detach_errors[debug-pg17-False] or test_ancestor_detach_branched_from[debug-pg17-False-False-earlier] or test_timeline_detach_ancestor_interrupted_by_deletion[debug-pg17-delete_timeline-False] or test_timeline_detach_ancestor_interrupted_by_deletion[debug-pg17-delete_tenant-False] or test_timeline_ancestor_detach_errors[debug-pg17-True] or test_retried_detach_ancestor_after_failed_reparenting[debug-pg17] or test_timeline_detach_ancestor_interrupted_by_deletion[debug-pg17-delete_timeline-True] or test_sharded_tad_interleaved_after_partial_success[debug-pg17-delete_reparentable_timeline] or test_sharded_tad_interleaved_after_partial_success[debug-pg17-create_reparentable_timeline] or test_retryable_500_hit_through_storcon_during_timeline_detach_ancestor[debug-pg17] or test_timeline_is_deleted_before_timeline_detach_ancestor_completes[debug-pg17] or test_sharded_timeline_detach_ancestor[debug-pg17] or test_gc_blocking_by_timeline[debug-pg17-False] or test_gc_blocking_by_timeline[debug-pg17-True] or test_timeline_size_createdropdb[debug-pg17] or test_timeline_physical_size_post_compaction[debug-pg17] or test_timeline_size_quota[debug-pg17] or test_timeline_physical_size_post_checkpoint[debug-pg17] or test_timeline_logical_size_task_priority[debug-pg17] or test_timeline_size_metrics[debug-pg17] or test_timeline_size_quota_on_startup[debug-pg17] or test_timeline_physical_size_post_gc[debug-pg17] or test_timeline_initial_logical_size_calculation_cancellation[debug-pg17-tenant_detach] or test_timeline_initial_logical_size_calculation_cancellation[debug-pg17-timeline_delete] or test_timeline_physical_size_init[debug-pg17] or test_ondemand_activation[debug-pg17] or test_eager_attach_does_not_queue_up[debug-pg17] or test_lazy_attach_activation[debug-pg17-endpoint] or test_lazy_attach_activation[debug-pg17-branch] or test_lazy_attach_activation[debug-pg17-delete] or test_timeline_size[debug-pg17] or test_tenant_physical_size[debug-pg17] or test_truncate[debug-pg17] or test_twophase_nonzero_epoch[debug-pg17] or test_twophase[debug-pg17] or test_twophase_at_wal_segment_start[debug-pg17] or test_unlogged[debug-pg17] or test_vm_bit_clear_on_heap_lock_blackbox[debug-pg17] or test_vm_bit_clear[debug-pg17] or test_vm_bit_clear_on_heap_lock_whitebox[debug-pg17] or test_check_visibility_map[debug-pg17] or test_broker[debug-pg17] or test_wal_removal[debug-pg17-False] or test_wal_removal[debug-pg17-True] or test_wal_backup[debug-pg17] or test_s3_wal_replay[debug-pg17] or test_many_timelines[debug-pg17] or test_late_init[debug-pg17] or test_restarts[debug-pg17] or test_sync_safekeepers[debug-pg17] or test_timeline_status[debug-pg17-False] or test_timeline_status[debug-pg17-True] or test_start_replication_term[debug-pg17] or test_sk_auth[debug-pg17] or test_restart_endpoint_wal_page_boundary[debug-pg17] or test_restart_endpoint_after_switch_wal[debug-pg17] or test_lagging_sk[debug-pg17] or test_wp_graceful_shutdown[debug-pg17] or test_peer_recovery[debug-pg17] or test_pull_timeline_term_change[debug-pg17] or test_s3_eviction[debug-pg17-0.2-False] or test_pull_timeline_partial_segment_integrity[debug-pg17] or test_timeline_copy[debug-pg17-0] or test_delete_force[debug-pg17-False] or test_replace_safekeeper[debug-pg17] or test_delete_force[debug-pg17-True] or test_pull_timeline[debug-pg17-False] or test_pull_timeline[debug-pg17-True] or test_delete_timeline_under_load[debug-pg17] or test_pull_timeline_gc[debug-pg17] or test_pull_timeline_while_evicted[debug-pg17] or test_membership_api[debug-pg17] or test_timeline_copy[debug-pg17-100000] or test_timeline_copy[debug-pg17-100] or test_timeline_copy[debug-pg17-500000] or test_term_bump[debug-pg17] or test_broker_discovery[debug-pg17] or test_patch_control_file[debug-pg17] or test_s3_eviction[debug-pg17-0.0-False] or test_backup_partial_reset[debug-pg17] or test_s3_eviction[debug-pg17-0.2-True] or test_s3_eviction[debug-pg17-0.0-True] or test_vm_truncate[debug-pg17] or test_unstable_extensions_installation[debug-pg17] or test_restarts_under_load[debug-pg17] or test_restarts_frequent_checkpoints[debug-pg17] or test_compute_restarts[debug-pg17] or test_concurrent_computes[debug-pg17] or test_unavailability[debug-pg17] or test_wal_truncation[debug-pg17] or test_recovery_uncommitted[debug-pg17] or test_race_conditions[debug-pg17] or test_segment_init_failure[debug-pg17-vanilla] or test_segment_init_failure[debug-pg17-interpreted] or test_pageserver_lsn_wait_error_safekeeper_stop[debug-pg17] or test_pageserver_lsn_wait_error_start[debug-pg17] or test_walredo_not_left_behind_on_detach[debug-pg17] or test_wal_restore[debug-pg17] or test_wal_restore_initdb[debug-pg17] or test_wal_restore_http[debug-pg17-False] or test_wal_restore_http[debug-pg17-True]"
Flaky tests (3)

Postgres 17

Test coverage report is not available

The comment gets automatically updated with the latest test results
9b081a8 at 2025-01-24T23:21:29.465Z :recycle:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@skyzh