TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

ZFS for Dummies

414 点作者 giis超过 1 年前

16 条评论

istjohn超过 1 年前
I&#x27;m getting started with ZFS just now. The learning curve is steeper than I expected. I would love to have a dumbed down wrapper that made the common case dead-simple. For example:<p>- Use sane defaults for pool creation. ashift=12, lz4 compression, xattr=sa, acltype=posixacl, and atime=off. Don&#x27;t even ask me.<p>- Make encryption just on or off instead of offering five or six options<p>- Generate the encryption key for me, set up the systemd service to decrypt the pool at start up, and prompt me to back up the key somewhere<p>- `zfs list` should show if a dataset is mounted or not, if it is encrypted or not, and if the encryption key is loaded or not<p>- No recursive datasets and use {pool}:{dataset} instead of {pool}&#x2F;{dataset} to maintain a clear distinction between pools and datasets.<p>- Don&#x27;t make me name pools or snapshots. Assign pools the name {hostname}-[A-Z]. Name snapshots {pool name}_{datetime created} and give them numerical shortcuts so I never have to type that all out<p>- Don&#x27;t make me type disk IDs when creating pools. Store metadata on the disk so ZFS doesn&#x27;t get confused if I set up a pool with `&#x2F;dev&#x2F;sda` and `&#x2F;dev&#x2F;sdb` references and then shuffle around the drives<p>- Always use `pv` to show progress<p>- Automatically set up weekly scrubs<p>- Automatically set up hourly&#x2F;daily&#x2F;weekly&#x2F;monthly snapshots and snapshot pruning<p>- If I send to a disk without a pool, ask for confirmation and then create a new single disk pool for me with the same settings as on the sending pool<p>- collapse `zpool` and `zfs` into a single command<p>- Automatically use `--raw` when sending encrypted datasets, default to `--replicate` when sending, and use `-I` whenever possible when sending<p>- Provide an obvious way to mount and navigate a snapshot dataset instead of hiding the snapshot filesystem in a hidden directory
评论 #37389535 未加载
评论 #37389553 未加载
评论 #37389174 未加载
评论 #37391200 未加载
评论 #37513593 未加载
评论 #37390048 未加载
评论 #37389470 未加载
评论 #37389152 未加载
评论 #37389290 未加载
评论 #37388915 未加载
评论 #37395855 未加载
评论 #37388984 未加载
评论 #37391232 未加载
评论 #37392636 未加载
评论 #37397042 未加载
评论 #37397048 未加载
评论 #37393557 未加载
评论 #37390575 未加载
vermaden超过 1 年前
Other useful things about ZFS:<p>- get to know the difference between <i>zpool-attach(8)</i> and <i>zpool-replace(8)</i>.<p>- this one will tell you where your space is used:<p><pre><code> # zfs list -t all -o space NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV USEDCHILD (...) </code></pre> - <i>ZFS Boot Environments</i> is the best feature to protect your OS before major changes&#x2F;upgrades<p>--- this may be useful for a start: <a href="https:&#x2F;&#x2F;is.gd&#x2F;BECTL" rel="nofollow noreferrer">https:&#x2F;&#x2F;is.gd&#x2F;BECTL</a><p>- this command will tell you all history about ZFS pool config and its changes:<p><pre><code> # zpool history poolname History for &#x27;poolname&#x27;: 2023-06-20.14:03:08 zpool create poolname ada0p1 2023-06-20.14:03:08 zpool set autotrim=on poolname 2023-06-20.14:03:08 zfs set atime=off poolname 2023-06-20.14:03:08 zfs set compression=zstd poolname 2023-06-20.14:03:08 zfs set recordsize=1m poolname (...) </code></pre> - the guide misses one important info:<p><pre><code> --- you can create 3-way mirror - requires 3 disks and 2 may fail - still no data lost --- you can create 4-way mirror - requires 4 disks and 3 may fail - still no data lost --- you can create N-way mirror - requires N disks and N-1 may fail - still no data lost (useful when data is most important and you do not have that many slots&#x2F;disks)</code></pre>
评论 #37394766 未加载
customizable超过 1 年前
We have been running a large multi-TB PostgreSQL database on ZFS for years now. ZFS makes it super easy to do backups, create test environments from past snapshots, and saves a lot of disk space thanks to built-in compression. In case anyone is interested, you can read our experience at <a href="https:&#x2F;&#x2F;lackofimagination.org&#x2F;2022&#x2F;04&#x2F;our-experience-with-postgresql-on-zfs&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;lackofimagination.org&#x2F;2022&#x2F;04&#x2F;our-experience-with-po...</a>
评论 #37393021 未加载
qwertox超过 1 年前
FreeBSD&#x27;s Handbook on ZFS [0] and Aaron Toponce&#x27;s articles [1] were what helped me the most when getting started with ZFS<p>[0] <a href="https:&#x2F;&#x2F;docs.freebsd.org&#x2F;en&#x2F;books&#x2F;handbook&#x2F;zfs&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;docs.freebsd.org&#x2F;en&#x2F;books&#x2F;handbook&#x2F;zfs&#x2F;</a><p>[1] <a href="https:&#x2F;&#x2F;pthree.org&#x2F;2012&#x2F;04&#x2F;17&#x2F;install-zfs-on-debian-gnulinux&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;pthree.org&#x2F;2012&#x2F;04&#x2F;17&#x2F;install-zfs-on-debian-gnulinux...</a>
评论 #37389405 未加载
philsnow超过 1 年前
One of the diagrams under the bit about snapshotting has a typo reading &quot;snapthot&quot; and I immediately thought it was talking about instagram.<p>(I realize now after writing it that maybe snapchat should have occurred to me first, but I have never used it)
tomxor超过 1 年前
I recently rebuilt a load of infrastructure (mainly LAMP servers) and decided to back them all with ZFS on Linux for the benefit of efficient backup replication and encryption.<p>I&#x27;ve been using ZFS in combination with rsync for backups for a long time, so I was fairly comfortable with it... and it all worked out, but it was a way bigger time sink than I expected - because I wanted to do it right - and there is a <i>lot</i> of misleading advice on the web, particularly when it comes to running databases and replication.<p>For databases (you really should at minimum do basic tuning like block size alignment), by far the best resource I found for mariadb&#x2F;innoDB is from the lets encrypt people [0]. They give reasons for everything and cite multiple sources, which is gold. If you search around the web elsewhere you will find endless contradicting advice, anecdotes and myths that are accompanied with incomplete and baseless theories. Ultimately you should also test this stuff and understand everything you tune (it&#x27;s ok to decide to not tune something).<p>For replication, I can only recommend the man pages... yeah, really! ZFS gives you solid replication tools, but they are too agnostic, they are like git pluming, they don&#x27;t assume you&#x27;re going to be doing it over SSH (even though that&#x27;s almost always how it&#x27;s being used)... so you have to plug it together yourself, and this feels scary at first, especially because you probably want it to be automated, which means considering edge cases... which is why everyone runs to something like syncoid, but there&#x27;s something horrible I discovered with replication scripts like syncoid, which is that they don&#x27;t use ZFS&#x27;s send --replication mode! They try to reimplement it in perl, for &quot;greater flexibility&quot;, but incompletely. This is maddening when you are trying to test this stuff for the first time and find that all of the encryption roots break when you do a fresh restore, and not all dataset properties are automatically synced. ZFS takes care of all of this if you simply use the build in recursive &quot;replicate&quot; option. It&#x27;s not that hard to script manually once you commit to it, just keep it simple, don&#x27;t add a bunch of unnecessary crap into the pipeline like syncoid does, (they actually slow it down if you test), just use pv to monitor progress and it will fly.<p>I might publish my replication scripts at some point because I feel like there are no good functional reference scripts for this stuff that deal with the basics without going nuts and reinventing replication badly like so many others.<p>[0] <a href="https:&#x2F;&#x2F;github.com&#x2F;letsencrypt&#x2F;openzfs-nvme-databases">https:&#x2F;&#x2F;github.com&#x2F;letsencrypt&#x2F;openzfs-nvme-databases</a>
评论 #37403739 未加载
评论 #37388281 未加载
guerby超过 1 年前
I started to use ZFS (on Linux) a few years ago and it went smoothly.<p>My only surprise was volblocksize default which is pretty bad for most RAIDZ configuration: you need to increase it to avoid loosing 50% of raw disk space...<p>Articles touching this topic :<p><a href="https:&#x2F;&#x2F;jro.io&#x2F;nas&#x2F;#overhead" rel="nofollow noreferrer">https:&#x2F;&#x2F;jro.io&#x2F;nas&#x2F;#overhead</a><p><a href="https:&#x2F;&#x2F;openzfs.github.io&#x2F;openzfs-docs&#x2F;Basic%20Concepts&#x2F;RAIDZ.html" rel="nofollow noreferrer">https:&#x2F;&#x2F;openzfs.github.io&#x2F;openzfs-docs&#x2F;Basic%20Concepts&#x2F;RAID...</a><p><a href="https:&#x2F;&#x2F;www.delphix.com&#x2F;blog&#x2F;zfs-raidz-stripe-width-or-how-i-learned-stop-worrying-and-love-raidz" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.delphix.com&#x2F;blog&#x2F;zfs-raidz-stripe-width-or-how-i...</a><p>And you end up on one of the ZFS &quot;spreadsheet&quot; out there:<p>ZFS overhead calc.xlsx <a href="https:&#x2F;&#x2F;docs.google.com&#x2F;spreadsheets&#x2F;d&#x2F;1tf4qx1aMJp8Lo_R6gpT689wTjHv6CGVElrPqTA0w_ZY" rel="nofollow noreferrer">https:&#x2F;&#x2F;docs.google.com&#x2F;spreadsheets&#x2F;d&#x2F;1tf4qx1aMJp8Lo_R6gpT6...</a><p>RAID-Z parity cost <a href="https:&#x2F;&#x2F;docs.google.com&#x2F;spreadsheets&#x2F;d&#x2F;1pdu_X2tR4ztF6_HLtJ-Dc4ZcwUdt6fkCjpnXxAEFlyA" rel="nofollow noreferrer">https:&#x2F;&#x2F;docs.google.com&#x2F;spreadsheets&#x2F;d&#x2F;1pdu_X2tR4ztF6_HLtJ-D...</a>
评论 #37398111 未加载
tweetle_beetle超过 1 年前
Might not remember the details correctly but when I was younger and stupider I read a lot about how great one of the open source NAS OSs (FreeNAS?) and ZFS were from fervent fans. I bought a very low spec second hand HP micro server on eBay and jumped straight in without really knowing what I was doing. I asked a few questions on the community forum but the vast majority of answers were &quot;Have you read the documentation?!&quot; &quot;Do you have enough RAM?!&quot;.<p>The documentation in question was a PowerPoint presentation with difficult to read styling, somewhat evangelical language, lots of assumptions about knowledge and it was not regularly updated. It was vague on how much RAM was required, mainly just focused on having as much as possible. Needless to say I ignored all the red flags about the technology, the hype and my own knowledge and lost a load of data. Lots of lessons learnt.
评论 #37389759 未加载
unethical_ban超过 1 年前
Some additional points for posterity, in case it isn&#x27;t driven home here:<p>- All redundancy in ZFS is built in the vdev layer. Zpools are created with one or more vdevs, and <i>no matter what</i>, if you lose any single vdev in a zpool, the zpool is permanently destroyed.<p>- Historically RAIDZs (parity RAIDs) cannot be expanded by adding disks. The only way to grow a RAIDZ is to replace each disk in the array one at a time with a larger disk (and hope no disks fail during the rebuild). So in my very amateur opinion, I would only consider doing a RAIDZ if it is something like a RAIDZ2 or 3 with a large number of disks. For n&lt;=6 and if the budget can stand it, I would do several mirrored vdevs. (Again as an amateur I am less familiar with RW performance metrics of various RAIDs so do more research for prod).
评论 #37403845 未加载
mastax超过 1 年前
I&#x27;ve run into a ZFS problem I don&#x27;t understand. I have a zpool where zpool status prints out a list of detected errors, never in files or `&lt;metadata&gt;` but in snapshots (and hex numbers that I assume are deleted snapshots). If I delete the listed errored snapshots and run zpool scrub twice the errors disappear and the scrub finds no errors. Zpool status never listed any errors for any of the devices.<p>So there aren&#x27;t any errors in files. There aren&#x27;t any errors in devices. There aren&#x27;t any errors detected in scrub(?). And yet at runtime I get a dozen new &quot;errors&quot; showing up in zpool status per day. How?
评论 #37389939 未加载
评论 #37392831 未加载
totetsu超过 1 年前
Nice. My gotchas form using zfs on my personal laptop with Ubuntu.<p>- if you want to copy files for example and connect your drive to another system and mount your zpool there, it sets some pool membership value on the file system and when you put it back in your system it won’t boot unless you set it back. Which involved chroot<p>- the default settings I had made snapshot every time I apt installed something, because that snap shot included my home drive when I deleted big files thereafter I didn’t get any free space back until i figued out what was going on and arbitrarily deleted some old snapshots<p>- you can’t just make a swap file and use it,
评论 #37387917 未加载
评论 #37388104 未加载
idatum超过 1 年前
I need 3 stores to feel I&#x27;m keeping safe years of digital family photos. 1) I have a live (local) FreeBSD ZFS server running for backups and snapshots; 2 pairs of mirrored physical drives 2) I have a USB device that takes 2 mirrored drives to recv ZFS snapshots from #1; I store that vdev backup in a safe place 3) I backup entire datasets to cloud storage from off-prem using rclone.<p>It&#x27;s #3 where I need to do some more research&#x2F;work. I need to spend some time sending snapshots&#x2F;diffs to cloud blob storage and make sure I can restore. Yes, I know there is rsync.net.<p>Any experiences to share?
评论 #37388996 未加载
评论 #37388575 未加载
评论 #37388999 未加载
评论 #37389478 未加载
评论 #37388264 未加载
评论 #37388530 未加载
评论 #37392114 未加载
asicsp超过 1 年前
See also: <a href="https:&#x2F;&#x2F;jro.io&#x2F;truenas&#x2F;openzfs&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;jro.io&#x2F;truenas&#x2F;openzfs&#x2F;</a> (discussed here: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=34318734">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=34318734</a>)
crawsome超过 1 年前
&quot;Also read up on the zpool add command.&quot;<p>Haha, The only part of maintenance that I need to look up every time I do it is replacing a faulty hard drive.<p>Even this guide skips that.
znpy超过 1 年前
is there an equivalent &quot;btrfs for dummies&quot; ?
dontupvoteme超过 1 年前
As nice as the technology is as long as there&#x27;s the potential of a Damoclean license issue I&#x27;ll always feel hesitant around ZFS.<p>(Hey looks like it&#x27;s a sore spot!)
评论 #37387994 未加载
评论 #37388192 未加载
评论 #37388655 未加载
评论 #37388396 未加载
评论 #37387925 未加载
评论 #37387831 未加载
评论 #37392804 未加载
评论 #37388191 未加载