Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

3525 persistent l2arc. #1823

Closed
wants to merge 1 commit into from
Closed

3525 persistent l2arc. #1823

wants to merge 1 commit into from

Conversation

yshui
Copy link
Contributor

@yshui yshui commented Nov 1, 2013

Details: https://www.illumos.org/issues/3525

This patch compiles, but haven't been tested. I will when I get time.

If anyone spent their time to review this, that will be appreicated.

@yshui
Copy link
Contributor Author

yshui commented Nov 7, 2013

This patch seems to be pretty much functional now, although the numbers in arcstats are completely insane.

Still looking for the cause, any suggestions?

@yshui
Copy link
Contributor Author

yshui commented Nov 8, 2013

I think I fixed the stat numbers

@yshui
Copy link
Contributor Author

yshui commented Mar 6, 2014

I've been using this for several months. It seems to be mostly fine, but has lock-ups during boot which I can't locate in the code. Could anyone help me test and review the code?

@maxximino
Copy link
Contributor

Do you have stack traces obtained during the lockups? (SysRq keys!) It happened during pool import or after?

@DeHackEd DeHackEd mentioned this pull request Mar 29, 2014
Closed
@kernelOfTruth
Copy link
Contributor

any updated feedback on this ?

did anyone figure out why it locks up ?

@yshui about what kind of lock-ups are we talking about ? the system completely locks up/hardlocks ?

currently I'm also using an l2arc and its kind of counterproductive that it needs to refill anew after every reboot (I need to reboot often into windows, so it takes a while until l2arc is full and fully beneficial again)

@yshui
Copy link
Contributor Author

yshui commented Aug 22, 2014

Sorry, I won't be able to do it for now. Anyone who is interested should take this.

@yshui
Copy link
Contributor Author

yshui commented Aug 23, 2014

I rebased it again. It seems to work in my virtual machine.

For now I don't have other test environments.

@yshui
3525 persistent l2arc.
e566c01 
For details, see: https://www.illumos.org/issues/3525

v2: Change two KM_SLEEP to KM_PUSHPAGE.
v3: Change one more KM_SLEEP.
v4: Fix log buffer alignment in l2arc_dev_log_commit.
v5: Fix style.
v6: l2arc vdev can go away, remove ASSERT in l2arc_spa_rebuild_start.

Close #925

Ported-by: Yuxuan Shui <yshuiv7@gmail.com>
@edillmann
Copy link
Contributor

Hi

@yshui Could you give us a little test procedure ?

I did warm up the cache, then try an export / import after which I found an empty l2arc ?!

Any clues ?

@yshui
Copy link
Contributor Author

yshui commented Aug 25, 2014

@edillmann Can you post the content of /proc/spl/kstat/zfs/arcstats? It's probably a bug.

Thanks

@edillmann
Copy link
Contributor

5 1 0x01 100 4800 37707460085 172635741662342
name type data
hits 4 18921818
misses 4 8551113
demand_data_hits 4 15425744
demand_data_misses 4 6363149
demand_metadata_hits 4 3061031
demand_metadata_misses 4 1067668
prefetch_data_hits 4 314951
prefetch_data_misses 4 1079956
prefetch_metadata_hits 4 120092
prefetch_metadata_misses 4 40340
mru_hits 4 2850452
mru_ghost_hits 4 194513
mfu_hits 4 15656677
mfu_ghost_hits 4 165640
deleted 4 1579518
recycle_miss 4 145650
mutex_miss 4 1605
evict_skip 4 2166954
evict_l2_cached 4 20595342848
evict_l2_eligible 4 7234583552
evict_l2_ineligible 4 3904925184
hash_elements 4 1262105
hash_elements_max 4 1657944
hash_collisions 4 1375951
hash_chains 4 151441
hash_chain_max 4 5
p 4 1244673024
c 4 8147483648
c_min 4 4194304
c_max 4 8147483648
size 4 8147408816
hdr_size 4 288200216
data_size 4 5928275968
meta_size 4 1496782848
other_size 4 272240848
anon_size 4 2199552
anon_evict_data 4 0
anon_evict_metadata 4 0
mru_size 4 1242272256
mru_evict_data 4 68616704
mru_evict_metadata 4 852506624
mru_ghost_size 4 6904673280
mru_ghost_evict_data 4 6498217984
mru_ghost_evict_metadata 4 406455296
mfu_size 4 6180587008
mfu_evict_data 4 5859364352
mfu_evict_metadata 4 235177984
mfu_ghost_size 4 1239119360
mfu_ghost_evict_data 4 1239119360
mfu_ghost_evict_metadata 4 0
l2_hits 4 376377
l2_misses 4 8172893
l2_feeds 4 172979
l2_rw_clash 4 11
l2_read_bytes 4 2792800768
l2_write_bytes 4 15593026222
l2_writes_sent 4 15557
l2_writes_done 4 15557
l2_writes_error 4 0
l2_writes_hdr_miss 4 12
l2_evict_lock_retry 4 0
l2_evict_reading 4 0
l2_free_on_write 4 21
l2_abort_lowmem 4 0
l2_cksum_bad 4 0
l2_io_error 4 0
l2_size 4 10736604160
l2_asize 4 8531538944
l2_hdr_size 4 161908936
l2_compress_successes 4 588148
l2_compress_zeros 4 0
l2_compress_failures 4 293739
l2_log_writes 4 1653
l2_log_avg_size 4 59072
l2_data_to_meta_ratio 4 235
l2_rebuild_successes 4 0
l2_rebuild_unsupported 4 0
l2_rebuild_timeout 4 0
l2_rebuild_io_errors 4 0
l2_rebuild_cksum_errors 4 0
l2_rebuild_loop_errors 4 0
l2_rebuild_lowmem 4 0
l2_rebuild_psize 4 0
l2_rebuild_bufs 4 0
l2_rebuild_bufs_precached 4 0
l2_rebuild_size 4 0
l2_rebuild_logs 4 0
memory_throttle_count 4 0
duplicate_buffers 4 0
duplicate_buffers_size 4 0
duplicate_reads 4 0
memory_direct_count 4 0
memory_indirect_count 4 0
arc_no_grow 4 0
arc_tempreserve 4 0
arc_loaned_bytes 4 0
arc_prune 4 0
arc_meta_used 4 2219132848
arc_meta_limit 4 6110612736
arc_meta_max 4 2709351800

@yshui
Copy link
Contributor Author

yshui commented Aug 25, 2014

@edillmann Is this the result after you export/import the pool?

@edillmann
Copy link
Contributor

@yshui yes, it's after export / import
i also try remove / add of cache device : same result

@edillmann
Copy link
Contributor

@yshui is there anything I can do to help you to track down this bug ?

@yshui
Copy link
Contributor Author

yshui commented Aug 31, 2014

@edillmann This is weird. Even if the rebuild failed, at least one of these should be non-zero:

l2_rebuild_successes 4 0
l2_rebuild_unsupported 4 0
l2_rebuild_timeout 4 0
l2_rebuild_io_errors 4 0
l2_rebuild_cksum_errors 4 0
l2_rebuild_loop_errors 4 0
l2_rebuild_lowmem 4 0

I'll see if I can reproduce this.

@edillmann
Copy link
Contributor

I'm tried a vanilla 3.16.1 kernel, with only this patch and obtain the same result

@edillmann
Copy link
Contributor

@yshui I tried all this again from scratch, fresh pull, rebase against master, and it work's :-)

I think I had some weird build environnment

After export / import a get the following stats:

l2_rebuild_successes 4 2
l2_rebuild_unsupported 4 4
l2_rebuild_timeout 4 0
l2_rebuild_io_errors 4 0
l2_rebuild_cksum_errors 4 0
l2_rebuild_loop_errors 4 0
l2_rebuild_lowmem 4 0
l2_rebuild_psize 4 934959104
l2_rebuild_bufs 4 8184
l2_rebuild_bufs_precached 4 114
l2_rebuild_size 4 915367936
l2_rebuild_logs 4 8

@edillmann
Copy link
Contributor

@yshui rebase to master can be found at https://github.com/edillmann/zfs/tree/illumos-3525

@behlendorf
Copy link
Contributor

@edillmann if you opened a pull request with the latest change we could get some testing result from it. Or @yshui could refresh this one.

@edillmann edillmann mentioned this pull request Sep 4, 2014
Closed
@yshui yshui closed this Sep 7, 2014
@yshui yshui deleted the illumos-3525 branch September 7, 2014 06:08
@yshui
Copy link
Contributor Author

yshui commented Sep 7, 2014

Sorry I closed this pull request by accidentally deleting the branch. I'll create a new one.

@yshui yshui mentioned this pull request Aug 15, 2018
Closed
behlendorf pushed a commit that referenced this pull request Apr 10, 2020
@gamanakis
Persistent L2ARC
77f6826 
This commit makes the L2ARC persistent across reboots. We implement
a light-weight persistent L2ARC metadata structure that allows L2ARC
contents to be recovered after a reboot. This significantly eases the
impact a reboot has on read performance on systems with large caches.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: George Wilson <gwilson@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Co-authored-by: Saso Kiselkov <skiselkov@gmail.com>
Co-authored-by: Jorgen Lundman <lundman@lundman.net>
Co-authored-by: George Amanakis <gamanakis@gmail.com>
Ported-by: Yuxuan Shui <yshuiv7@gmail.com>
Signed-off-by: George Amanakis <gamanakis@gmail.com>
Closes #925 
Closes #1823 
Closes #2672 
Closes #3744 
Closes #9582
jsai20 pushed a commit to jsai20/zfs that referenced this pull request Mar 30, 2021
@gamanakis @jsai20
Persistent L2ARC
eb3ffe4 
This commit makes the L2ARC persistent across reboots. We implement
a light-weight persistent L2ARC metadata structure that allows L2ARC
contents to be recovered after a reboot. This significantly eases the
impact a reboot has on read performance on systems with large caches.

Reviewed-by: Matthew Ahrens <mahrens@delphix.com>
Reviewed-by: George Wilson <gwilson@delphix.com>
Reviewed-by: Ryan Moeller <ryan@iXsystems.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Co-authored-by: Saso Kiselkov <skiselkov@gmail.com>
Co-authored-by: Jorgen Lundman <lundman@lundman.net>
Co-authored-by: George Amanakis <gamanakis@gmail.com>
Ported-by: Yuxuan Shui <yshuiv7@gmail.com>
Signed-off-by: George Amanakis <gamanakis@gmail.com>
Closes openzfs#925 
Closes openzfs#1823 
Closes openzfs#2672 
Closes openzfs#3744 
Closes openzfs#9582
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Type: Feature Feature request or new feature
Projects
None yet
Milestone
0.6.4
Development

Successfully merging this pull request may close these issues.
5 participants
@yshui @maxximino @kernelOfTruth @edillmann @behlendorf