From postmaster@james.hungrycats.org Tue Jun 23 18:20:57 2015 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on james.vpn7.hungrycats.org X-Spam-Status: No, score=-5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,T_RP_MATCHES_RCVD X-Spam-Hint: -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high trust [209.132.180.67 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.132.180.67 listed in wl.mailspike.net] 0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different -0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-Spam-DCCB: X-Spam-DCCR: X-Spam-Langs: X-Spam-Pyzor: Reported 0 times. X-Spam-RBL: [127.0.0.18] [209.132.180.67] [999 bl-ckh-le.kernel.org., 10 vger.kernel.org.] [127.0.9.3] X-Spam-Scores: HEADER_FROM_DIFFERENT_DOMAINS=0.001,RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01,RCVD_IN_MSPIKE_WL=-0.01,T_RP_MATCHES_RCVD=-0.01 X-Spam-Subtests: __BODY_TEXT_LINE,__BODY_TEXT_LINE,__BODY_TEXT_LINE, __DOS_HAS_ANY_URI,__DOS_HAS_LIST_ID,__DOS_RCVD_TUE,__DOS_RELAYED_EXT, __FROM_FULL_NAME,__HAS_ANY_EMAIL,__HAS_ANY_URI,__HAS_CC,__HAS_DATE,__HAS_FROM, __HAS_LIST_ID,__HAS_MESSAGE_ID,__HAS_MSGID,__HAS_RCVD,__HAS_SENDER, __HAS_SUBJECT,__HAS_TO,__HAS_URI,__HAS_XMAIL,__HAS_X_MAILER, __HAS_X_MAILING_LIST,__KB_DATE_CONTAINS_TAB,__LAST_EXTERNAL_RELAY_NO_AUTH, __LAST_UNTRUSTED_RELAY_NO_AUTH,__LOCAL_PP_NONPPURL,__MISSING_REF, __MISSING_REPLY,__ML1,__ML2,__ML5,__ML_TURNS_SP_TO_TAB,__MSGID_OK_DIGITS, __MSGID_OK_HOST,__MSOE_MID_WRONG_CASE,__NONEMPTY_BODY,__NOT_A_PERSON, __RCVD_IN_DNSWL,__RCVD_IN_MSPIKE_L,__RP_MATCHES_RCVD,__SANE_MSGID, __SUBSCRIPTION_INFO,__TAB_IN_FROM,__TOCC_EXISTS,__URI_MAILTO,__VIA_ML, __freemail_safe X-Original-To: ce3g8jdj@umail.furryterror.org X-Envelope-Mail-From: linux-btrfs-owner@vger.kernel.org Return-Path: linux-btrfs-owner@vger.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by james.kirk.hungrycats.org (Postfix) with ESMTP id 3094516B222C for ; Tue, 23 Jun 2015 17:30:11 -0400 (EDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754988AbbFWV2v (ORCPT ); Tue, 23 Jun 2015 17:28:51 -0400 Received: from cantor2.suse.de ([195.135.220.15]:52101 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755001AbbFWV2t (ORCPT ); Tue, 23 Jun 2015 17:28:49 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 12939AC57; Tue, 23 Jun 2015 21:28:48 +0000 (UTC) From: Mark Fasheh To: Chris Mason Cc: Josef Bacik , David Sterba , linux-btrfs@vger.kernel.org Subject: [PATCH 0/5] btrfs: dedupe fixes, features V3 Date: Tue, 23 Jun 2015 14:28:35 -0700 Message-Id: <1435094920-22442-1-git-send-email-mfasheh@suse.de> X-Mailer: git-send-email 2.1.2 Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-GeoIP-Hint: 209.132.180.67 crc!United'States/NC/Raleigh dma!560 code!US postcode!27606 country!United'States region!NC city!Raleigh code3!USA area!919 latitude!35.746299743652344 longitude!'78.7238998413086 lat1!36 lon1!'79 lat90!0 lon90!'90 lat2!36 lon2!'78 lat45!45 lon45!'90 lat3!36 lon3!'78 lat30!30 lon30!'90 lat5!35 lon5!'80 lat18!36 lon18!'72 lat6!36 lon6!'78 lat15!30 lon15!'75 lat9!36 lon9!'81 lat10!40 lon10!'80 X-DNS-Hint: 209.132.180.67 209 132 180 67 vger.kernel.org vger kernel org X-ASN-Hint: Origin: AS31976 Net: 209.132.180.0/23 US arin 2004_01_22 Desc: US arin 2004_02_03 REDHAT_0 _ Red Hat, Inc.,US X-ASN-Hint: Origin: AS22753 Net: 209.132.176.0/20 US arin 2004_01_22 Desc: US arin 2001_11_15 REDHAT_0 _ Red Hat, Inc.,US X-RBL-Hint: 209.132.180.67 rbl 9.3.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 9.Y.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 3.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 18.rep.mailspike.net.0 a=0\;t=0\;s=0\;r=3 X-RBL-Hint: 209.132.180.67 rbl 9.3.list.dnswl.org.0 9.Y.list.dnswl.org.0 3.list.dnswl.org.0 18.rep.mailspike.net.0 X-GeoIP-Hint-1: 195.135.220.15 crc!Germany// dma!0 code!DE postcode! country!Germany region! city! code3!DEU area!0 latitude!51.0 longitude!9.0 lat1!51 lon1!9 lat90!90 lon90!0 lat2!52 lon2!10 lat45!45 lon45!0 lat3!51 lon3!9 lat30!60 lon30!0 lat5!50 lon5!10 lat18!54 lon18!18 lat6!54 lon6!12 lat15!45 lon15!15 lat9!54 lon9!9 lat10!50 lon10!10 X-DNS-Hint-1: 195.135.220.15 195 135 220 15 cantor2.suse.de cantor2 suse de X-ASN-Hint-1: Origin: AS29298 Net: 195.135.220.0/22 DE ripencc 2003_07_22 Desc: DE ripencc 2003_07_24 AS29298 SUSE LINUX GmbH,DE X-RBL-Hint: 195.135.220.15 rbl 9.3.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 9.Y.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 3.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 9.3.list.dnswl.org.1 9.Y.list.dnswl.org.1 3.list.dnswl.org.1 X-GeoIP-Hint-2: 195.135.220.254 crc!Germany// dma!0 code!DE postcode! country!Germany region! city! code3!DEU area!0 latitude!51.0 longitude!9.0 lat1!51 lon1!9 lat90!90 lon90!0 lat2!52 lon2!10 lat45!45 lon45!0 lat3!51 lon3!9 lat30!60 lon30!0 lat5!50 lon5!10 lat18!54 lon18!18 lat6!54 lon6!12 lat15!45 lon15!15 lat9!54 lon9!9 lat10!50 lon10!10 X-DNS-Hint-2: 195.135.220.254 195 135 220 254 X-RBL-Hint: 195.135.220.254 rbl none.2 X-Relay-Hint: ipv4 209.132.180.67 X-DSPAM-Bypass: relay=209.132.180.67 spam=492 ham=216185 bypass=62938 discard=0 since=2011-08-10 result=ham X-DSPAM-Result: Innocent X-DSPAM-Processed: Tue Jun 23 18:21:38 2015 X-DSPAM-Confidence: 0.9996 X-DSPAM-Improbability: 1 in 237303 chance of being spam X-DSPAM-Probability: 0.0000 X-DSPAM-Factors: 27, X-Mailer*git-send-email, 0.00010, Subject*PATCH, 0.00010, X-Spam-Subtests*__HAS_ANY_URI+__HAS_CC, 0.00011, X-Spam-Subtests*__HAS_ANY_EMAIL+#+__HAS_CC, 0.00013, X-Spam-Subtests*__HAS_ANY_URI+#+__HAS_DATE, 0.00015, Cc*David, 0.00016, X-Spam-Subtests*__HAS_CC, 0.00018, X-Spam-Subtests*__HAS_ANY_EMAIL+#+#+__HAS_DATE, 0.00018, X-Spam-Subtests*__HAS_CC+__HAS_DATE, 0.00023, git+#+org, 0.00024, Received*by+mx2.suse.de, 0.00025, Received*mx2.suse.de+Postfix, 0.00025, Received*mx2.suse.de+#+with, 0.00025, Received*mx2.suse.de, 0.00025, Received*mx2.suse.de, 0.00025, Received*mx2.suse.de+#+#+#+id, 0.00025, Received*195.135.220.15, 0.00025, https+git, 0.00025, Received*cantor2.suse.de, 0.00027, git+#+#+#+git, 0.00028, Cc*suse.cz, 0.00028, X-Spam-Subtests*__HAS_CC+#+__HAS_FROM, 0.00028, patch+adds, 0.00028, Received*mx2.suse.de+#+#+ESMTP, 0.00030, X-RBL-Hint*195.135.220.254, 0.00031, X-RBL-Hint*195.135.220.254+rbl, 0.00031, Received*from+#+#+195.135.220.254, 0.00031 Content-Length: 1828 Lines: 47 Hi Chris, The following patches are based on top of my patch titled "btrfs: Handle unaligned length in extent_same" which you have in your 'integration-4.2' branch: https://git.kernel.org/cgit/linux/kernel/git/mason/linux-btrfs.git/commit/?id=e1d227a42ea2b4664f94212bd1106b9a3413ffb8 The series contains three fixes and two features. The first patch in the series fixes a bug where we were sometimes passing the aligned length to our comparison function. We actually can stop at the user passed length for this as we don't need to compare data past i_size (and we only align if the extents go out to i_size). The 2nd patch fixes a deadlock between btrfs readpage and btrfs_extent_same. This was reported on the list some months ago - basically we had the page and extent locking reversed. My patch fixes up the locking to be in the right order. The 3rd patch fixes a deadlocks in clone() (wrt extent-same) which David found while reviewing my fixes. I also found that clone doesn't lock extent ranges in any particular order which could obvioulsy be a problem so that is fixed there too. The last two patches add features which have been requested often by users - the 4th adds the ability to dedupe within the same inode, and the last patch adds a dedupe flag to avoid mtime updates (this helps with backup software). These patches have been tested with the 'btrfs-extent-same' tool that can be found at: https://github.com/markfasheh/duperemove/blob/nomtime/btrfs-extent-same.c Changes from V2 to V3: - better flag naming in patch #5, thanks to David Changes from V1 to V2: - added patches 3-5 Thanks, --Mark -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html From postmaster@james.hungrycats.org Tue Jun 23 18:20:57 2015 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on james.vpn7.hungrycats.org X-Spam-Status: No, score=-5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,T_RP_MATCHES_RCVD X-Spam-Hint: -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high trust [209.132.180.67 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.132.180.67 listed in wl.mailspike.net] 0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different -0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-Spam-DCCB: X-Spam-DCCR: X-Spam-Langs: X-Spam-Pyzor: Reported 0 times. X-Spam-RBL: [127.0.9.3] [10 vger.kernel.org., 999 bl-ckh-le.kernel.org.] [127.0.0.18] [209.132.180.67] X-Spam-Scores: HEADER_FROM_DIFFERENT_DOMAINS=0.001,RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01,RCVD_IN_MSPIKE_WL=-0.01,T_RP_MATCHES_RCVD=-0.01 X-Spam-Subtests: __BODY_TEXT_LINE,__BODY_TEXT_LINE,__BODY_TEXT_LINE, __DOS_HAS_ANY_URI,__DOS_HAS_LIST_ID,__DOS_RCVD_TUE,__DOS_RELAYED_EXT, __FROM_FULL_NAME,__HAS_ANY_EMAIL,__HAS_ANY_URI,__HAS_CC,__HAS_DATE,__HAS_FROM, __HAS_IN_REPLY_TO,__HAS_LIST_ID,__HAS_MESSAGE_ID,__HAS_MSGID,__HAS_RCVD, __HAS_SENDER,__HAS_SUBJECT,__HAS_TO,__HAS_URI,__HAS_XMAIL,__HAS_X_MAILER, __HAS_X_MAILING_LIST,__HAS_X_REF,__KB_DATE_CONTAINS_TAB, __LAST_EXTERNAL_RELAY_NO_AUTH,__LAST_UNTRUSTED_RELAY_NO_AUTH, __LOCAL_PP_NONPPURL,__ML1,__ML2,__ML5,__ML_TURNS_SP_TO_TAB,__MSGID_OK_DIGITS, __MSGID_OK_HOST,__MSOE_MID_WRONG_CASE,__NONEMPTY_BODY,__NOT_A_PERSON, __RCVD_IN_DNSWL,__RCVD_IN_MSPIKE_L,__RP_MATCHES_RCVD,__SANE_MSGID, __SUBSCRIPTION_INFO,__TAB_IN_FROM,__THREADED,__TOCC_EXISTS,__URI_MAILTO, __URI_MAILTO,__URI_MAILTO,__VIA_ML,__freemail_safe X-Original-To: ce3g8jdj@umail.furryterror.org X-Envelope-Mail-From: linux-btrfs-owner@vger.kernel.org Return-Path: linux-btrfs-owner@vger.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by james.kirk.hungrycats.org (Postfix) with ESMTP id A1A2916B222E for ; Tue, 23 Jun 2015 17:30:11 -0400 (EDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754987AbbFWV2w (ORCPT ); Tue, 23 Jun 2015 17:28:52 -0400 Received: from cantor2.suse.de ([195.135.220.15]:52105 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932123AbbFWV2w (ORCPT ); Tue, 23 Jun 2015 17:28:52 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id ECA0FAC69; Tue, 23 Jun 2015 21:28:49 +0000 (UTC) From: Mark Fasheh To: Chris Mason Cc: Josef Bacik , David Sterba , linux-btrfs@vger.kernel.org, Mark Fasheh Subject: [PATCH 1/5] btrfs: pass unaligned length to btrfs_cmp_data() Date: Tue, 23 Jun 2015 14:28:36 -0700 Message-Id: <1435094920-22442-2-git-send-email-mfasheh@suse.de> X-Mailer: git-send-email 2.1.2 In-Reply-To: <1435094920-22442-1-git-send-email-mfasheh@suse.de> References: <1435094920-22442-1-git-send-email-mfasheh@suse.de> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-GeoIP-Hint: 209.132.180.67 crc!United'States/NC/Raleigh dma!560 code!US postcode!27606 country!United'States region!NC city!Raleigh code3!USA area!919 latitude!35.746299743652344 longitude!'78.7238998413086 lat1!36 lon1!'79 lat90!0 lon90!'90 lat2!36 lon2!'78 lat45!45 lon45!'90 lat3!36 lon3!'78 lat30!30 lon30!'90 lat5!35 lon5!'80 lat18!36 lon18!'72 lat6!36 lon6!'78 lat15!30 lon15!'75 lat9!36 lon9!'81 lat10!40 lon10!'80 X-DNS-Hint: 209.132.180.67 209 132 180 67 vger.kernel.org vger kernel org X-ASN-Hint: Origin: AS22753 Net: 209.132.176.0/20 US arin 2004_01_22 Desc: US arin 2001_11_15 REDHAT_0 _ Red Hat, Inc.,US X-ASN-Hint: Origin: AS31976 Net: 209.132.176.0/21 US arin 2004_01_22 Desc: US arin 2004_02_03 REDHAT_0 _ Red Hat, Inc.,US X-RBL-Hint: 209.132.180.67 rbl 9.3.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 9.Y.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 3.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 18.rep.mailspike.net.0 a=0\;t=0\;s=0\;r=3 X-RBL-Hint: 209.132.180.67 rbl 9.3.list.dnswl.org.0 9.Y.list.dnswl.org.0 3.list.dnswl.org.0 18.rep.mailspike.net.0 X-GeoIP-Hint-1: 195.135.220.15 crc!Germany// dma!0 code!DE postcode! country!Germany region! city! code3!DEU area!0 latitude!51.0 longitude!9.0 lat1!51 lon1!9 lat90!90 lon90!0 lat2!52 lon2!10 lat45!45 lon45!0 lat3!51 lon3!9 lat30!60 lon30!0 lat5!50 lon5!10 lat18!54 lon18!18 lat6!54 lon6!12 lat15!45 lon15!15 lat9!54 lon9!9 lat10!50 lon10!10 X-DNS-Hint-1: 195.135.220.15 195 135 220 15 cantor2.suse.de cantor2 suse de X-ASN-Hint-1: Origin: AS29298 Net: 195.135.220.0/22 DE ripencc 2003_07_22 Desc: DE ripencc 2003_07_24 AS29298 SUSE LINUX GmbH,DE X-RBL-Hint: 195.135.220.15 rbl 9.3.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 9.Y.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 3.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 9.3.list.dnswl.org.1 9.Y.list.dnswl.org.1 3.list.dnswl.org.1 X-GeoIP-Hint-2: 195.135.220.254 crc!Germany// dma!0 code!DE postcode! country!Germany region! city! code3!DEU area!0 latitude!51.0 longitude!9.0 lat1!51 lon1!9 lat90!90 lon90!0 lat2!52 lon2!10 lat45!45 lon45!0 lat3!51 lon3!9 lat30!60 lon30!0 lat5!50 lon5!10 lat18!54 lon18!18 lat6!54 lon6!12 lat15!45 lon15!15 lat9!54 lon9!9 lat10!50 lon10!10 X-DNS-Hint-2: 195.135.220.254 195 135 220 254 X-RBL-Hint: 195.135.220.254 rbl none.2 X-Relay-Hint: ipv4 209.132.180.67 X-DSPAM-Bypass: relay=209.132.180.67 spam=492 ham=216184 bypass=62938 discard=0 since=2011-08-10 result=ham X-DSPAM-Result: Innocent X-DSPAM-Processed: Tue Jun 23 18:21:37 2015 X-DSPAM-Confidence: 0.9997 X-DSPAM-Improbability: 1 in 333510 chance of being spam X-DSPAM-Probability: 0.0000 X-DSPAM-Signature: 5589dbf18171851546477 X-DSPAM-Factors: 27, git+a, 0.00010, 7+#+#+static, 0.00010, file+changed, 0.00010, 100644+a, 0.00010, changed+#+insertions, 0.00010, 1+#+changed, 0.00010, index+#+#+#+a, 0.00010, c+#+#+#+changed, 0.00010, 100644, 0.00010, insertions+#+#+#+git, 0.00010, c+index, 0.00010, diff+git, 0.00010, 1+#+diff, 0.00010, changed+#+#+1, 0.00010, index+#+#+100644, 0.00010, 1+#+#+#+insertions, 0.00010, 1+file, 0.00010, c+#+#+#+100644, 0.00010, insertions+1, 0.00010, int+#+#+#+struct, 0.00010, Signed+#+by, 0.00010, changed+#+#+#+deletion, 0.00010, 1+#+#+git, 0.00010, diff+#+a, 0.00010, 1+deletion, 0.00010, Signed+off, 0.00010, static+int, 0.00010 Content-Length: 1085 Lines: 31 In the case that we dedupe the tail of a file, we might expand the dedupe len out to the end of our last block. We don't want to compare data past i_size however, so pass the original length to btrfs_cmp_data(). Signed-off-by: Mark Fasheh Reviewed-by: David Sterba --- fs/btrfs/ioctl.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 2d24ff4..2deea1f 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -2933,7 +2933,8 @@ static int btrfs_extent_same(struct inode *src, u64 loff, u64 olen, goto out_unlock; } - ret = btrfs_cmp_data(src, loff, dst, dst_loff, len); + /* pass original length for comparison so we stay within i_size */ + ret = btrfs_cmp_data(src, loff, dst, dst_loff, olen); if (ret == 0) ret = btrfs_clone(src, dst, loff, olen, len, dst_loff); -- 2.1.2 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html From postmaster@james.hungrycats.org Tue Jun 23 18:21:42 2015 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on james.vpn7.hungrycats.org X-Spam-Status: No, score=-5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,T_RP_MATCHES_RCVD X-Spam-Hint: -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high trust [209.132.180.67 listed in list.dnswl.org] 0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different -0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.132.180.67 listed in wl.mailspike.net] -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-Spam-DCCB: X-Spam-DCCR: X-Spam-Langs: X-Spam-Pyzor: Reported 0 times. X-Spam-RBL: [999 bl-ckh-le.kernel.org., 10 vger.kernel.org.] [127.0.9.3] [209.132.180.67] [127.0.0.18] X-Spam-Scores: HEADER_FROM_DIFFERENT_DOMAINS=0.001,RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01,RCVD_IN_MSPIKE_WL=-0.01,T_RP_MATCHES_RCVD=-0.01 X-Spam-Subtests: __BODY_TEXT_LINE,__BODY_TEXT_LINE,__BODY_TEXT_LINE, __DOS_HAS_ANY_URI,__DOS_HAS_LIST_ID,__DOS_RCVD_TUE,__DOS_RELAYED_EXT, __FROM_FULL_NAME,__HAS_ANY_EMAIL,__HAS_ANY_URI,__HAS_CC,__HAS_DATE,__HAS_FROM, __HAS_IN_REPLY_TO,__HAS_LIST_ID,__HAS_MESSAGE_ID,__HAS_MSGID,__HAS_RCVD, __HAS_SENDER,__HAS_SUBJECT,__HAS_TO,__HAS_URI,__HAS_XMAIL,__HAS_X_MAILER, __HAS_X_MAILING_LIST,__HAS_X_REF,__KB_DATE_CONTAINS_TAB, __LAST_EXTERNAL_RELAY_NO_AUTH,__LAST_UNTRUSTED_RELAY_NO_AUTH, __LOCAL_PP_NONPPURL,__ML1,__ML2,__ML5,__ML_TURNS_SP_TO_TAB,__MSGID_OK_DIGITS, __MSGID_OK_HOST,__MSOE_MID_WRONG_CASE,__NONEMPTY_BODY,__NOT_A_PERSON, __RCVD_IN_DNSWL,__RCVD_IN_MSPIKE_L,__RP_MATCHES_RCVD,__SANE_MSGID, __SUBSCRIPTION_INFO,__TAB_IN_FROM,__THREADED,__TOCC_EXISTS,__URI_MAILTO, __URI_MAILTO,__URI_MAILTO,__VIA_ML,__freemail_safe X-Original-To: ce3g8jdj@umail.furryterror.org X-Envelope-Mail-From: linux-btrfs-owner@vger.kernel.org Return-Path: linux-btrfs-owner@vger.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by james.kirk.hungrycats.org (Postfix) with ESMTP id A33B816B22FF for ; Tue, 23 Jun 2015 17:31:30 -0400 (EDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932849AbbFWVaN (ORCPT ); Tue, 23 Jun 2015 17:30:13 -0400 Received: from cantor2.suse.de ([195.135.220.15]:52113 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755013AbbFWV2z (ORCPT ); Tue, 23 Jun 2015 17:28:55 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 22C36AC69; Tue, 23 Jun 2015 21:28:54 +0000 (UTC) From: Mark Fasheh To: Chris Mason Cc: Josef Bacik , David Sterba , linux-btrfs@vger.kernel.org, Mark Fasheh Subject: [PATCH 3/5] btrfs: fix clone / extent-same deadlocks Date: Tue, 23 Jun 2015 14:28:38 -0700 Message-Id: <1435094920-22442-4-git-send-email-mfasheh@suse.de> X-Mailer: git-send-email 2.1.2 In-Reply-To: <1435094920-22442-1-git-send-email-mfasheh@suse.de> References: <1435094920-22442-1-git-send-email-mfasheh@suse.de> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-GeoIP-Hint: 209.132.180.67 crc!United'States/NC/Raleigh dma!560 code!US postcode!27606 country!United'States region!NC city!Raleigh code3!USA area!919 latitude!35.746299743652344 longitude!'78.7238998413086 lat1!36 lon1!'79 lat90!0 lon90!'90 lat2!36 lon2!'78 lat45!45 lon45!'90 lat3!36 lon3!'78 lat30!30 lon30!'90 lat5!35 lon5!'80 lat18!36 lon18!'72 lat6!36 lon6!'78 lat15!30 lon15!'75 lat9!36 lon9!'81 lat10!40 lon10!'80 X-DNS-Hint: 209.132.180.67 209 132 180 67 vger.kernel.org vger kernel org X-ASN-Hint: Origin: AS31976 Net: 209.132.176.0/21 US arin 2004_01_22 Desc: US arin 2004_02_03 REDHAT_0 _ Red Hat, Inc.,US X-RBL-Hint: 209.132.180.67 rbl 9.3.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 9.Y.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 3.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 18.rep.mailspike.net.0 a=0\;t=0\;s=0\;r=3 X-RBL-Hint: 209.132.180.67 rbl 9.3.list.dnswl.org.0 9.Y.list.dnswl.org.0 3.list.dnswl.org.0 18.rep.mailspike.net.0 X-GeoIP-Hint-1: 195.135.220.15 crc!Germany// dma!0 code!DE postcode! country!Germany region! city! code3!DEU area!0 latitude!51.0 longitude!9.0 lat1!51 lon1!9 lat90!90 lon90!0 lat2!52 lon2!10 lat45!45 lon45!0 lat3!51 lon3!9 lat30!60 lon30!0 lat5!50 lon5!10 lat18!54 lon18!18 lat6!54 lon6!12 lat15!45 lon15!15 lat9!54 lon9!9 lat10!50 lon10!10 X-DNS-Hint-1: 195.135.220.15 195 135 220 15 cantor2.suse.de cantor2 suse de X-ASN-Hint-1: Origin: AS29298 Net: 195.135.220.0/22 DE ripencc 2003_07_22 Desc: DE ripencc 2003_07_24 AS29298 SUSE LINUX GmbH,DE X-RBL-Hint: 195.135.220.15 rbl 9.3.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 9.Y.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 3.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 9.3.list.dnswl.org.1 9.Y.list.dnswl.org.1 3.list.dnswl.org.1 X-GeoIP-Hint-2: 195.135.220.254 crc!Germany// dma!0 code!DE postcode! country!Germany region! city! code3!DEU area!0 latitude!51.0 longitude!9.0 lat1!51 lon1!9 lat90!90 lon90!0 lat2!52 lon2!10 lat45!45 lon45!0 lat3!51 lon3!9 lat30!60 lon30!0 lat5!50 lon5!10 lat18!54 lon18!18 lat6!54 lon6!12 lat15!45 lon15!15 lat9!54 lon9!9 lat10!50 lon10!10 X-DNS-Hint-2: 195.135.220.254 195 135 220 254 X-RBL-Hint: 195.135.220.254 rbl none.2 X-Relay-Hint: ipv4 209.132.180.67 X-DSPAM-Bypass: relay=209.132.180.67 spam=492 ham=216186 bypass=62938 discard=0 since=2011-08-10 result=ham X-DSPAM-Result: Innocent X-DSPAM-Processed: Tue Jun 23 18:22:28 2015 X-DSPAM-Confidence: 0.9997 X-DSPAM-Improbability: 1 in 333510 chance of being spam X-DSPAM-Probability: 0.0000 X-DSPAM-Factors: 27, git+a, 0.00010, file+changed, 0.00010, 100644+a, 0.00010, changed+#+insertions, 0.00010, 1+#+changed, 0.00010, index+#+#+#+a, 0.00010, c+#+#+#+changed, 0.00010, 100644, 0.00010, insertions+#+#+#+git, 0.00010, c+index, 0.00010, diff+git, 0.00010, insertions+#+deletions, 0.00010, deletions+diff, 0.00010, index+#+#+100644, 0.00010, deletions, 0.00010, 1+#+#+#+insertions, 0.00010, 1+file, 0.00010, c+#+#+#+100644, 0.00010, changed+#+#+#+deletions, 0.00010, Signed+#+by, 0.00010, 7+static, 0.00010, 7+static, 0.00010, static+void, 0.00010, static+void, 0.00010, diff+#+a, 0.00010, deletions+#+git, 0.00010, Signed+off, 0.00010 Content-Length: 3524 Lines: 103 Clone and extent same lock their source and target inodes in opposite order. In addition to this, the range locking in clone doesn't take ordering into account. Fix this by having clone use the same locking helpers as btrfs-extent-same. In addition, I do a small cleanup of the locking helpers, removing a case (both inodes being the same) which was poorly accounted for and never actually used by the callers. Signed-off-by: Mark Fasheh Reviewed-by: David Sterba --- fs/btrfs/ioctl.c | 34 ++++++++-------------------------- 1 file changed, 8 insertions(+), 26 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index b899584..8d6887d 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -2831,8 +2831,7 @@ static void btrfs_double_inode_lock(struct inode *inode1, struct inode *inode2) swap(inode1, inode2); mutex_lock_nested(&inode1->i_mutex, I_MUTEX_PARENT); - if (inode1 != inode2) - mutex_lock_nested(&inode2->i_mutex, I_MUTEX_CHILD); + mutex_lock_nested(&inode2->i_mutex, I_MUTEX_CHILD); } static void btrfs_double_extent_unlock(struct inode *inode1, u64 loff1, @@ -2850,8 +2849,7 @@ static void btrfs_double_extent_lock(struct inode *inode1, u64 loff1, swap(loff1, loff2); } lock_extent_range(inode1, loff1, len); - if (inode1 != inode2) - lock_extent_range(inode2, loff2, len); + lock_extent_range(inode2, loff2, len); } struct cmp_pages { @@ -3713,13 +3711,7 @@ static noinline long btrfs_ioctl_clone(struct file *file, unsigned long srcfd, goto out_fput; if (!same_inode) { - if (inode < src) { - mutex_lock_nested(&inode->i_mutex, I_MUTEX_PARENT); - mutex_lock_nested(&src->i_mutex, I_MUTEX_CHILD); - } else { - mutex_lock_nested(&src->i_mutex, I_MUTEX_PARENT); - mutex_lock_nested(&inode->i_mutex, I_MUTEX_CHILD); - } + btrfs_double_inode_lock(src, inode); } else { mutex_lock(&src->i_mutex); } @@ -3769,8 +3761,7 @@ static noinline long btrfs_ioctl_clone(struct file *file, unsigned long srcfd, lock_extent_range(src, lock_start, lock_len); } else { - lock_extent_range(src, off, len); - lock_extent_range(inode, destoff, len); + btrfs_double_extent_lock(src, off, inode, destoff, len); } ret = btrfs_clone(src, inode, off, olen, len, destoff); @@ -3781,9 +3772,7 @@ static noinline long btrfs_ioctl_clone(struct file *file, unsigned long srcfd, unlock_extent(&BTRFS_I(src)->io_tree, lock_start, lock_end); } else { - unlock_extent(&BTRFS_I(src)->io_tree, off, off + len - 1); - unlock_extent(&BTRFS_I(inode)->io_tree, destoff, - destoff + len - 1); + btrfs_double_extent_unlock(src, off, inode, destoff, len); } /* * Truncate page cache pages so that future reads will see the cloned @@ -3792,17 +3781,10 @@ static noinline long btrfs_ioctl_clone(struct file *file, unsigned long srcfd, truncate_inode_pages_range(&inode->i_data, destoff, PAGE_CACHE_ALIGN(destoff + len) - 1); out_unlock: - if (!same_inode) { - if (inode < src) { - mutex_unlock(&src->i_mutex); - mutex_unlock(&inode->i_mutex); - } else { - mutex_unlock(&inode->i_mutex); - mutex_unlock(&src->i_mutex); - } - } else { + if (!same_inode) + btrfs_double_inode_unlock(src, inode); + else mutex_unlock(&src->i_mutex); - } out_fput: fdput(src_file); out_drop_write: -- 2.1.2 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html From postmaster@james.hungrycats.org Tue Jun 23 18:21:52 2015 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on james.vpn7.hungrycats.org X-Spam-Status: No, score=-5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,T_RP_MATCHES_RCVD X-Spam-Hint: -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high trust [209.132.180.67 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.132.180.67 listed in wl.mailspike.net] 0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different -0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-Spam-DCCB: X-Spam-DCCR: X-Spam-Langs: X-Spam-Pyzor: Reported 0 times. X-Spam-RBL: [127.0.0.18] [209.132.180.67] [10 vger.kernel.org., 999 bl-ckh-le.kernel.org.] [127.0.9.3] X-Spam-Scores: HEADER_FROM_DIFFERENT_DOMAINS=0.001,RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01,RCVD_IN_MSPIKE_WL=-0.01,T_RP_MATCHES_RCVD=-0.01 X-Spam-Subtests: __BODY_TEXT_LINE,__BODY_TEXT_LINE,__BODY_TEXT_LINE, __DOS_HAS_ANY_URI,__DOS_HAS_LIST_ID,__DOS_RCVD_TUE,__DOS_RELAYED_EXT, __FROM_FULL_NAME,__HAS_ANY_EMAIL,__HAS_ANY_URI,__HAS_CC,__HAS_DATE,__HAS_FROM, __HAS_IN_REPLY_TO,__HAS_LIST_ID,__HAS_MESSAGE_ID,__HAS_MSGID,__HAS_RCVD, __HAS_SENDER,__HAS_SUBJECT,__HAS_TO,__HAS_URI,__HAS_XMAIL,__HAS_X_MAILER, __HAS_X_MAILING_LIST,__HAS_X_REF,__KB_DATE_CONTAINS_TAB, __LAST_EXTERNAL_RELAY_NO_AUTH,__LAST_UNTRUSTED_RELAY_NO_AUTH, __LOCAL_PP_NONPPURL,__ML1,__ML2,__ML5,__ML_TURNS_SP_TO_TAB,__MSGID_OK_DIGITS, __MSGID_OK_HOST,__MSOE_MID_WRONG_CASE,__NONEMPTY_BODY,__NOT_A_PERSON, __RCVD_IN_DNSWL,__RCVD_IN_MSPIKE_L,__RP_MATCHES_RCVD,__SANE_MSGID, __SUBSCRIPTION_INFO,__TAB_IN_FROM,__THREADED,__TOCC_EXISTS,__URI_MAILTO, __URI_MAILTO,__URI_MAILTO,__VIA_ML,__freemail_safe X-Original-To: ce3g8jdj@umail.furryterror.org X-Envelope-Mail-From: linux-btrfs-owner@vger.kernel.org Return-Path: linux-btrfs-owner@vger.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by james.kirk.hungrycats.org (Postfix) with ESMTP id C686F16B2304 for ; Tue, 23 Jun 2015 17:31:31 -0400 (EDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932994AbbFWVaP (ORCPT ); Tue, 23 Jun 2015 17:30:15 -0400 Received: from cantor2.suse.de ([195.135.220.15]:52120 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755009AbbFWV25 (ORCPT ); Tue, 23 Jun 2015 17:28:57 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 02F56AC8C; Tue, 23 Jun 2015 21:28:56 +0000 (UTC) From: Mark Fasheh To: Chris Mason Cc: Josef Bacik , David Sterba , linux-btrfs@vger.kernel.org, Mark Fasheh Subject: [PATCH 4/5] btrfs: allow dedupe of same inode Date: Tue, 23 Jun 2015 14:28:39 -0700 Message-Id: <1435094920-22442-5-git-send-email-mfasheh@suse.de> X-Mailer: git-send-email 2.1.2 In-Reply-To: <1435094920-22442-1-git-send-email-mfasheh@suse.de> References: <1435094920-22442-1-git-send-email-mfasheh@suse.de> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-GeoIP-Hint: 209.132.180.67 crc!United'States/NC/Raleigh dma!560 code!US postcode!27606 country!United'States region!NC city!Raleigh code3!USA area!919 latitude!35.746299743652344 longitude!'78.7238998413086 lat1!36 lon1!'79 lat90!0 lon90!'90 lat2!36 lon2!'78 lat45!45 lon45!'90 lat3!36 lon3!'78 lat30!30 lon30!'90 lat5!35 lon5!'80 lat18!36 lon18!'72 lat6!36 lon6!'78 lat15!30 lon15!'75 lat9!36 lon9!'81 lat10!40 lon10!'80 X-DNS-Hint: 209.132.180.67 209 132 180 67 vger.kernel.org vger kernel org X-ASN-Hint: Origin: AS31976 Net: 209.132.180.0/23 US arin 2004_01_22 Desc: US arin 2004_02_03 REDHAT_0 _ Red Hat, Inc.,US X-ASN-Hint: Origin: AS22753 Net: 209.132.176.0/20 US arin 2004_01_22 Desc: US arin 2001_11_15 REDHAT_0 _ Red Hat, Inc.,US X-RBL-Hint: 209.132.180.67 rbl 9.3.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 9.Y.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 3.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 18.rep.mailspike.net.0 a=0\;t=0\;s=0\;r=3 X-RBL-Hint: 209.132.180.67 rbl 9.3.list.dnswl.org.0 9.Y.list.dnswl.org.0 3.list.dnswl.org.0 18.rep.mailspike.net.0 X-GeoIP-Hint-1: 195.135.220.15 crc!Germany// dma!0 code!DE postcode! country!Germany region! city! code3!DEU area!0 latitude!51.0 longitude!9.0 lat1!51 lon1!9 lat90!90 lon90!0 lat2!52 lon2!10 lat45!45 lon45!0 lat3!51 lon3!9 lat30!60 lon30!0 lat5!50 lon5!10 lat18!54 lon18!18 lat6!54 lon6!12 lat15!45 lon15!15 lat9!54 lon9!9 lat10!50 lon10!10 X-DNS-Hint-1: 195.135.220.15 195 135 220 15 cantor2.suse.de cantor2 suse de X-ASN-Hint-1: Origin: AS29298 Net: 195.135.220.0/22 DE ripencc 2003_07_22 Desc: DE ripencc 2003_07_24 AS29298 SUSE LINUX GmbH,DE X-RBL-Hint: 195.135.220.15 rbl 9.3.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 9.Y.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 3.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 9.3.list.dnswl.org.1 9.Y.list.dnswl.org.1 3.list.dnswl.org.1 X-GeoIP-Hint-2: 195.135.220.254 crc!Germany// dma!0 code!DE postcode! country!Germany region! city! code3!DEU area!0 latitude!51.0 longitude!9.0 lat1!51 lon1!9 lat90!90 lon90!0 lat2!52 lon2!10 lat45!45 lon45!0 lat3!51 lon3!9 lat30!60 lon30!0 lat5!50 lon5!10 lat18!54 lon18!18 lat6!54 lon6!12 lat15!45 lon15!15 lat9!54 lon9!9 lat10!50 lon10!10 X-DNS-Hint-2: 195.135.220.254 195 135 220 254 X-RBL-Hint: 195.135.220.254 rbl none.2 X-Relay-Hint: ipv4 209.132.180.67 X-DSPAM-Bypass: relay=209.132.180.67 spam=492 ham=216187 bypass=62938 discard=0 since=2011-08-10 result=ham X-DSPAM-Result: Innocent X-DSPAM-Processed: Tue Jun 23 18:23:06 2015 X-DSPAM-Confidence: 0.9997 X-DSPAM-Improbability: 1 in 333510 chance of being spam X-DSPAM-Probability: 0.0000 X-DSPAM-Factors: 27, git+a, 0.00010, file+changed, 0.00010, 100644+a, 0.00010, changed+#+insertions, 0.00010, 1+#+changed, 0.00010, index+#+#+#+a, 0.00010, c+#+#+#+changed, 0.00010, 100644, 0.00010, insertions+#+#+#+git, 0.00010, c+index, 0.00010, diff+git, 0.00010, insertions+#+deletions, 0.00010, deletions+diff, 0.00010, index+#+#+100644, 0.00010, deletions, 0.00010, 1+#+#+#+insertions, 0.00010, 1+file, 0.00010, c+#+#+#+100644, 0.00010, changed+#+#+#+deletions, 0.00010, int+#+#+#+struct, 0.00010, int+#+#+#+struct, 0.00010, Signed+#+by, 0.00010, This+patch, 0.00010, diff+#+a, 0.00010, deletions+#+git, 0.00010, Signed+off, 0.00010, static+int, 0.00010 Content-Length: 3852 Lines: 129 clone() supports cloning within an inode so extent-same can do the same now. This patch fixes up the locking in extent-same to know about the single-inode case. In addition to that, we add a check for overlapping ranges, which clone does not allow. Signed-off-by: Mark Fasheh Reviewed-by: David Sterba --- fs/btrfs/ioctl.c | 76 ++++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 60 insertions(+), 16 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 8d6887d..83f4679 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -2979,27 +2979,61 @@ static int btrfs_extent_same(struct inode *src, u64 loff, u64 olen, int ret; u64 len = olen; struct cmp_pages cmp; + int same_inode = 0; + u64 same_lock_start = 0; + u64 same_lock_len = 0; - /* - * btrfs_clone() can't handle extents in the same file - * yet. Once that works, we can drop this check and replace it - * with a check for the same inode, but overlapping extents. - */ if (src == dst) - return -EINVAL; + same_inode = 1; if (len == 0) return 0; - btrfs_double_inode_lock(src, dst); + if (same_inode) { + mutex_lock(&src->i_mutex); - ret = extent_same_check_offsets(src, loff, &len, olen); - if (ret) - goto out_unlock; + ret = extent_same_check_offsets(src, loff, &len, olen); + if (ret) + goto out_unlock; - ret = extent_same_check_offsets(dst, dst_loff, &len, olen); - if (ret) - goto out_unlock; + /* + * Single inode case wants the same checks, except we + * don't want our length pushed out past i_size as + * comparing that data range makes no sense. + * + * extent_same_check_offsets() will do this for an + * unaligned length at i_size, so catch it here and + * reject the request. + * + * This effectively means we require aligned extents + * for the single-inode case, whereas the other cases + * allow an unaligned length so long as it ends at + * i_size. + */ + if (len != olen) { + ret = -EINVAL; + goto out_unlock; + } + + /* Check for overlapping ranges */ + if (dst_loff + len > loff && dst_loff < loff + len) { + ret = -EINVAL; + goto out_unlock; + } + + same_lock_start = min_t(u64, loff, dst_loff); + same_lock_len = max_t(u64, loff, dst_loff) + len - same_lock_start; + } else { + btrfs_double_inode_lock(src, dst); + + ret = extent_same_check_offsets(src, loff, &len, olen); + if (ret) + goto out_unlock; + + ret = extent_same_check_offsets(dst, dst_loff, &len, olen); + if (ret) + goto out_unlock; + } /* don't make the dst file partly checksummed */ if ((BTRFS_I(src)->flags & BTRFS_INODE_NODATASUM) != @@ -3012,18 +3046,28 @@ static int btrfs_extent_same(struct inode *src, u64 loff, u64 olen, if (ret) goto out_unlock; - btrfs_double_extent_lock(src, loff, dst, dst_loff, len); + if (same_inode) + lock_extent_range(src, same_lock_start, same_lock_len); + else + btrfs_double_extent_lock(src, loff, dst, dst_loff, len); /* pass original length for comparison so we stay within i_size */ ret = btrfs_cmp_data(src, loff, dst, dst_loff, olen, &cmp); if (ret == 0) ret = btrfs_clone(src, dst, loff, olen, len, dst_loff); - btrfs_double_extent_unlock(src, loff, dst, dst_loff, len); + if (same_inode) + unlock_extent(&BTRFS_I(src)->io_tree, same_lock_start, + same_lock_start + same_lock_len - 1); + else + btrfs_double_extent_unlock(src, loff, dst, dst_loff, len); btrfs_cmp_data_free(&cmp); out_unlock: - btrfs_double_inode_unlock(src, dst); + if (same_inode) + mutex_unlock(&src->i_mutex); + else + btrfs_double_inode_unlock(src, dst); return ret; } -- 2.1.2 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html From postmaster@james.hungrycats.org Tue Jun 23 18:23:09 2015 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on james.vpn7.hungrycats.org X-Spam-Status: No, score=-5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,T_RP_MATCHES_RCVD X-Spam-Hint: -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high trust [209.132.180.67 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.132.180.67 listed in wl.mailspike.net] 0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different -0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-Spam-DCCB: X-Spam-DCCR: X-Spam-Langs: X-Spam-Pyzor: Reported 0 times. X-Spam-RBL: [127.0.9.3] [999 bl-ckh-le.kernel.org., 10 vger.kernel.org.] [127.0.0.18] [209.132.180.67] X-Spam-Scores: HEADER_FROM_DIFFERENT_DOMAINS=0.001,RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01,RCVD_IN_MSPIKE_WL=-0.01,T_RP_MATCHES_RCVD=-0.01 X-Spam-Subtests: __BODY_TEXT_LINE,__BODY_TEXT_LINE,__BODY_TEXT_LINE, __DOS_HAS_ANY_URI,__DOS_HAS_LIST_ID,__DOS_RCVD_TUE,__DOS_RELAYED_EXT, __FROM_FULL_NAME,__HAS_ANY_EMAIL,__HAS_ANY_URI,__HAS_CC,__HAS_DATE,__HAS_FROM, __HAS_IN_REPLY_TO,__HAS_LIST_ID,__HAS_MESSAGE_ID,__HAS_MSGID,__HAS_RCVD, __HAS_SENDER,__HAS_SUBJECT,__HAS_TO,__HAS_URI,__HAS_XMAIL,__HAS_X_MAILER, __HAS_X_MAILING_LIST,__HAS_X_REF,__KB_DATE_CONTAINS_TAB, __LAST_EXTERNAL_RELAY_NO_AUTH,__LAST_UNTRUSTED_RELAY_NO_AUTH, __LOCAL_PP_NONPPURL,__ML1,__ML2,__ML5,__ML_TURNS_SP_TO_TAB,__MSGID_OK_DIGITS, __MSGID_OK_HOST,__MSOE_MID_WRONG_CASE,__NONEMPTY_BODY,__NOT_A_PERSON, __RCVD_IN_DNSWL,__RCVD_IN_MSPIKE_L,__RP_MATCHES_RCVD,__SANE_MSGID, __SUBSCRIPTION_INFO,__TAB_IN_FROM,__THREADED,__TOCC_EXISTS,__URI_MAILTO, __URI_MAILTO,__VIA_ML,__freemail_safe X-Original-To: ce3g8jdj@umail.furryterror.org X-Envelope-Mail-From: linux-btrfs-owner@vger.kernel.org Return-Path: linux-btrfs-owner@vger.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by james.kirk.hungrycats.org (Postfix) with ESMTP id C559816B23D5 for ; Tue, 23 Jun 2015 17:32:45 -0400 (EDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933264AbbFWVaQ (ORCPT ); Tue, 23 Jun 2015 17:30:16 -0400 Received: from cantor2.suse.de ([195.135.220.15]:52124 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755012AbbFWV27 (ORCPT ); Tue, 23 Jun 2015 17:28:59 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id D668DAC57; Tue, 23 Jun 2015 21:28:57 +0000 (UTC) From: Mark Fasheh To: Chris Mason Cc: Josef Bacik , David Sterba , linux-btrfs@vger.kernel.org, Mark Fasheh Subject: [PATCH 5/5] btrfs: add no_mtime flag to btrfs-extent-same Date: Tue, 23 Jun 2015 14:28:40 -0700 Message-Id: <1435094920-22442-6-git-send-email-mfasheh@suse.de> X-Mailer: git-send-email 2.1.2 In-Reply-To: <1435094920-22442-1-git-send-email-mfasheh@suse.de> References: <1435094920-22442-1-git-send-email-mfasheh@suse.de> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-GeoIP-Hint: 209.132.180.67 crc!United'States/NC/Raleigh dma!560 code!US postcode!27606 country!United'States region!NC city!Raleigh code3!USA area!919 latitude!35.746299743652344 longitude!'78.7238998413086 lat1!36 lon1!'79 lat90!0 lon90!'90 lat2!36 lon2!'78 lat45!45 lon45!'90 lat3!36 lon3!'78 lat30!30 lon30!'90 lat5!35 lon5!'80 lat18!36 lon18!'72 lat6!36 lon6!'78 lat15!30 lon15!'75 lat9!36 lon9!'81 lat10!40 lon10!'80 X-DNS-Hint: 209.132.180.67 209 132 180 67 vger.kernel.org vger kernel org X-ASN-Hint: Origin: AS22753 Net: 209.132.176.0/20 US arin 2004_01_22 Desc: US arin 2001_11_15 REDHAT_0 _ Red Hat, Inc.,US X-ASN-Hint: Origin: AS31976 Net: 209.132.176.0/21 US arin 2004_01_22 Desc: US arin 2004_02_03 REDHAT_0 _ Red Hat, Inc.,US X-RBL-Hint: 209.132.180.67 rbl 9.3.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 9.Y.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 3.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 18.rep.mailspike.net.0 a=0\;t=0\;s=0\;r=3 X-RBL-Hint: 209.132.180.67 rbl 9.3.list.dnswl.org.0 9.Y.list.dnswl.org.0 3.list.dnswl.org.0 18.rep.mailspike.net.0 X-GeoIP-Hint-1: 195.135.220.15 crc!Germany// dma!0 code!DE postcode! country!Germany region! city! code3!DEU area!0 latitude!51.0 longitude!9.0 lat1!51 lon1!9 lat90!90 lon90!0 lat2!52 lon2!10 lat45!45 lon45!0 lat3!51 lon3!9 lat30!60 lon30!0 lat5!50 lon5!10 lat18!54 lon18!18 lat6!54 lon6!12 lat15!45 lon15!15 lat9!54 lon9!9 lat10!50 lon10!10 X-DNS-Hint-1: 195.135.220.15 195 135 220 15 cantor2.suse.de cantor2 suse de X-ASN-Hint-1: Origin: AS29298 Net: 195.135.220.0/22 DE ripencc 2003_07_22 Desc: DE ripencc 2003_07_24 AS29298 SUSE LINUX GmbH,DE X-RBL-Hint: 195.135.220.15 rbl 9.3.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 9.Y.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 3.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 9.3.list.dnswl.org.1 9.Y.list.dnswl.org.1 3.list.dnswl.org.1 X-GeoIP-Hint-2: 195.135.220.254 crc!Germany// dma!0 code!DE postcode! country!Germany region! city! code3!DEU area!0 latitude!51.0 longitude!9.0 lat1!51 lon1!9 lat90!90 lon90!0 lat2!52 lon2!10 lat45!45 lon45!0 lat3!51 lon3!9 lat30!60 lon30!0 lat5!50 lon5!10 lat18!54 lon18!18 lat6!54 lon6!12 lat15!45 lon15!15 lat9!54 lon9!9 lat10!50 lon10!10 X-DNS-Hint-2: 195.135.220.254 195 135 220 254 X-RBL-Hint: 195.135.220.254 rbl none.2 X-Relay-Hint: ipv4 209.132.180.67 X-DSPAM-Bypass: relay=209.132.180.67 spam=492 ham=216190 bypass=62938 discard=0 since=2011-08-10 result=ham X-DSPAM-Result: Innocent X-DSPAM-Processed: Tue Jun 23 18:24:15 2015 X-DSPAM-Confidence: 0.9997 X-DSPAM-Improbability: 1 in 333510 chance of being spam X-DSPAM-Probability: 0.0000 X-DSPAM-Factors: 27, git+a, 0.00010, git+a, 0.00010, 7+#+#+static, 0.00010, 7+#+#+static, 0.00010, files+changed, 0.00010, 100644+a, 0.00010, 100644+a, 0.00010, files+#+#+insertions, 0.00010, changed+#+insertions, 0.00010, 2+#+changed, 0.00010, 6+#+#+static, 0.00010, 6+#+#+static, 0.00010, index+#+#+#+a, 0.00010, index+#+#+#+a, 0.00010, 100644, 0.00010, 100644, 0.00010, 2+#+#+#+insertions, 0.00010, insertions+#+#+#+git, 0.00010, linux+#+h, 0.00010, linux+#+h, 0.00010, h+#+#+files, 0.00010, c+index, 0.00010, h+index, 0.00010, diff+git, 0.00010, diff+git, 0.00010, insertions+#+deletions, 0.00010, const, 0.00010 Status: RO Content-Length: 6225 Lines: 173 One issue users have reported is that dedupe changes mtime on files, resulting in tools like rsync thinking that their contents have changed when in fact the data is exactly the same. Clone still wants an mtime change, so we special case this in the code. With this patch an application can pass the BTRFS_SAME_NO_MTIME flag to a dedupe request and the kernel will honor it by only changing ctime. I have an updated version of the btrfs-extent-same test program with a switch to provide this flag at the 'no_time' branch of: https://github.com/markfasheh/duperemove/ Signed-off-by: Mark Fasheh --- fs/btrfs/ioctl.c | 37 +++++++++++++++++++++++++------------ include/uapi/linux/btrfs.h | 5 ++++- 2 files changed, 29 insertions(+), 13 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 83f4679..ad717c8 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -87,7 +87,8 @@ struct btrfs_ioctl_received_subvol_args_32 { static int btrfs_clone(struct inode *src, struct inode *inode, - u64 off, u64 olen, u64 olen_aligned, u64 destoff); + u64 off, u64 olen, u64 olen_aligned, u64 destoff, + int flags); /* Mask out flags that are inappropriate for the given type of inode. */ static inline __u32 btrfs_mask_flags(umode_t mode, __u32 flags) @@ -2974,7 +2975,7 @@ static int extent_same_check_offsets(struct inode *inode, u64 off, u64 *plen, } static int btrfs_extent_same(struct inode *src, u64 loff, u64 olen, - struct inode *dst, u64 dst_loff) + struct inode *dst, u64 dst_loff, int flags) { int ret; u64 len = olen; @@ -3054,7 +3055,8 @@ static int btrfs_extent_same(struct inode *src, u64 loff, u64 olen, /* pass original length for comparison so we stay within i_size */ ret = btrfs_cmp_data(src, loff, dst, dst_loff, olen, &cmp); if (ret == 0) - ret = btrfs_clone(src, dst, loff, olen, len, dst_loff); + ret = btrfs_clone(src, dst, loff, olen, len, dst_loff, + flags); if (same_inode) unlock_extent(&BTRFS_I(src)->io_tree, same_lock_start, @@ -3088,6 +3090,7 @@ static long btrfs_ioctl_file_extent_same(struct file *file, u64 bs = BTRFS_I(src)->root->fs_info->sb->s_blocksize; bool is_admin = capable(CAP_SYS_ADMIN); u16 count; + int flags = 0; if (!(file->f_mode & FMODE_READ)) return -EINVAL; @@ -3139,6 +3142,11 @@ static long btrfs_ioctl_file_extent_same(struct file *file, if (!S_ISREG(src->i_mode)) goto out; + ret = -EINVAL; + if (same->flags & ~BTRFS_EXTENT_SAME_FLAG_MASK) + goto out; + flags = same->flags; + /* pre-format output fields to sane values */ for (i = 0; i < count; i++) { same->info[i].bytes_deduped = 0ULL; @@ -3164,7 +3172,8 @@ static long btrfs_ioctl_file_extent_same(struct file *file, info->status = -EACCES; } else { info->status = btrfs_extent_same(src, off, len, dst, - info->logical_offset); + info->logical_offset, + flags); if (info->status == 0) info->bytes_deduped += len; } @@ -3219,13 +3228,17 @@ static int clone_finish_inode_update(struct btrfs_trans_handle *trans, struct inode *inode, u64 endoff, const u64 destoff, - const u64 olen) + const u64 olen, + int flags) { struct btrfs_root *root = BTRFS_I(inode)->root; int ret; inode_inc_iversion(inode); - inode->i_mtime = inode->i_ctime = CURRENT_TIME; + if (flags & BTRFS_EXTENT_SAME_NO_MTIME) + inode->i_ctime = CURRENT_TIME; + else + inode->i_mtime = inode->i_ctime = CURRENT_TIME; /* * We round up to the block size at eof when determining which * extents to clone above, but shouldn't round up the file size. @@ -3310,13 +3323,13 @@ static void clone_update_extent_map(struct inode *inode, * @inode: Inode to clone to * @off: Offset within source to start clone from * @olen: Original length, passed by user, of range to clone - * @olen_aligned: Block-aligned value of olen, extent_same uses - * identical values here + * @olen_aligned: Block-aligned value of olen * @destoff: Offset within @inode to start clone + * @flags: clone flags from extent-same */ static int btrfs_clone(struct inode *src, struct inode *inode, const u64 off, const u64 olen, const u64 olen_aligned, - const u64 destoff) + const u64 destoff, int flags) { struct btrfs_root *root = BTRFS_I(inode)->root; struct btrfs_path *path = NULL; @@ -3640,7 +3653,7 @@ process_slot: root->sectorsize); ret = clone_finish_inode_update(trans, inode, last_dest_end, - destoff, olen); + destoff, olen, flags); if (ret) goto out; if (new_key.offset + datal >= destoff + len) @@ -3678,7 +3691,7 @@ process_slot: clone_update_extent_map(inode, trans, NULL, last_dest_end, destoff + len - last_dest_end); ret = clone_finish_inode_update(trans, inode, destoff + len, - destoff, olen); + destoff, olen, flags); } out: @@ -3808,7 +3821,7 @@ static noinline long btrfs_ioctl_clone(struct file *file, unsigned long srcfd, btrfs_double_extent_lock(src, off, inode, destoff, len); } - ret = btrfs_clone(src, inode, off, olen, len, destoff); + ret = btrfs_clone(src, inode, off, olen, len, destoff, 0); if (same_inode) { u64 lock_start = min_t(u64, off, destoff); diff --git a/include/uapi/linux/btrfs.h b/include/uapi/linux/btrfs.h index b6dec05..e2ab01e 100644 --- a/include/uapi/linux/btrfs.h +++ b/include/uapi/linux/btrfs.h @@ -342,11 +342,14 @@ struct btrfs_ioctl_same_extent_info { __u32 reserved; }; +#define BTRFS_EXTENT_SAME_NO_MTIME 0x1 +#define BTRFS_EXTENT_SAME_FLAG_MASK (BTRFS_EXTENT_SAME_NO_MTIME) + struct btrfs_ioctl_same_args { __u64 logical_offset; /* in - start of extent in source */ __u64 length; /* in - length of extent */ __u16 dest_count; /* in - total elements in info array */ - __u16 reserved1; + __u16 flags; /* in - see BTRFS_SAME_FLAGS */ __u32 reserved2; struct btrfs_ioctl_same_extent_info info[0]; }; -- 2.1.2 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html From postmaster@james.hungrycats.org Tue Jun 23 18:24:36 2015 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on james.vpn7.hungrycats.org X-Spam-Status: No, score=-5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,T_RP_MATCHES_RCVD X-Spam-Hint: -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high trust [209.132.180.67 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.132.180.67 listed in wl.mailspike.net] 0.0 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different -0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-Spam-DCCB: X-Spam-DCCR: X-Spam-Langs: X-Spam-Pyzor: Reported 0 times. X-Spam-RBL: [209.132.180.67] [127.0.0.18] [10 vger.kernel.org., 999 bl-ckh-le.kernel.org.] [127.0.9.3] X-Spam-Scores: HEADER_FROM_DIFFERENT_DOMAINS=0.001,RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01,RCVD_IN_MSPIKE_WL=-0.01,T_RP_MATCHES_RCVD=-0.01 X-Spam-Subtests: __BODY_TEXT_LINE,__BODY_TEXT_LINE,__BODY_TEXT_LINE, __DOS_HAS_ANY_URI,__DOS_HAS_LIST_ID,__DOS_RCVD_TUE,__DOS_RELAYED_EXT, __FROM_FULL_NAME,__HAS_ANY_EMAIL,__HAS_ANY_URI,__HAS_CC,__HAS_DATE,__HAS_FROM, __HAS_IN_REPLY_TO,__HAS_LIST_ID,__HAS_MESSAGE_ID,__HAS_MSGID,__HAS_RCVD, __HAS_SENDER,__HAS_SUBJECT,__HAS_TO,__HAS_URI,__HAS_XMAIL,__HAS_X_MAILER, __HAS_X_MAILING_LIST,__HAS_X_REF,__KB_DATE_CONTAINS_TAB, __LAST_EXTERNAL_RELAY_NO_AUTH,__LAST_UNTRUSTED_RELAY_NO_AUTH, __LOCAL_PP_NONPPURL,__ML1,__ML2,__ML5,__ML_TURNS_SP_TO_TAB,__MSGID_OK_DIGITS, __MSGID_OK_HOST,__MSOE_MID_WRONG_CASE,__NONEMPTY_BODY,__NOT_A_PERSON, __RCVD_IN_DNSWL,__RCVD_IN_MSPIKE_L,__RP_MATCHES_RCVD,__SANE_MSGID, __SUBSCRIPTION_INFO,__TAB_IN_FROM,__THREADED,__TOCC_EXISTS,__URI_MAILTO, __URI_MAILTO,__URI_MAILTO,__VIA_ML,__freemail_safe X-Original-To: ce3g8jdj@umail.furryterror.org X-Envelope-Mail-From: linux-btrfs-owner@vger.kernel.org Return-Path: linux-btrfs-owner@vger.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by james.kirk.hungrycats.org (Postfix) with ESMTP id C5E1B16B24A6 for ; Tue, 23 Jun 2015 17:34:09 -0400 (EDT) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932187AbbFWVaK (ORCPT ); Tue, 23 Jun 2015 17:30:10 -0400 Received: from cantor2.suse.de ([195.135.220.15]:52109 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755001AbbFWV2x (ORCPT ); Tue, 23 Jun 2015 17:28:53 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (charybdis-ext.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 0EF25AC57; Tue, 23 Jun 2015 21:28:52 +0000 (UTC) From: Mark Fasheh To: Chris Mason Cc: Josef Bacik , David Sterba , linux-btrfs@vger.kernel.org, Mark Fasheh Subject: [PATCH 2/5] btrfs: fix deadlock with extent-same and readpage Date: Tue, 23 Jun 2015 14:28:37 -0700 Message-Id: <1435094920-22442-3-git-send-email-mfasheh@suse.de> X-Mailer: git-send-email 2.1.2 In-Reply-To: <1435094920-22442-1-git-send-email-mfasheh@suse.de> References: <1435094920-22442-1-git-send-email-mfasheh@suse.de> Sender: linux-btrfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-btrfs@vger.kernel.org X-GeoIP-Hint: 209.132.180.67 crc!United'States/NC/Raleigh dma!560 code!US postcode!27606 country!United'States region!NC city!Raleigh code3!USA area!919 latitude!35.746299743652344 longitude!'78.7238998413086 lat1!36 lon1!'79 lat90!0 lon90!'90 lat2!36 lon2!'78 lat45!45 lon45!'90 lat3!36 lon3!'78 lat30!30 lon30!'90 lat5!35 lon5!'80 lat18!36 lon18!'72 lat6!36 lon6!'78 lat15!30 lon15!'75 lat9!36 lon9!'81 lat10!40 lon10!'80 X-DNS-Hint: 209.132.180.67 209 132 180 67 vger.kernel.org vger kernel org X-ASN-Hint: Origin: AS22753 Net: 209.132.176.0/20 US arin 2004_01_22 Desc: US arin 2001_11_15 REDHAT_0 _ Red Hat, Inc.,US X-ASN-Hint: Origin: AS31976 Net: 209.132.176.0/21 US arin 2004_01_22 Desc: US arin 2004_02_03 REDHAT_0 _ Red Hat, Inc.,US X-RBL-Hint: 209.132.180.67 rbl 9.3.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 9.Y.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 3.list.dnswl.org.0 kernel.org http://dnswl.org/s?s=2184 X-RBL-Hint: 209.132.180.67 rbl 18.rep.mailspike.net.0 a=0\;t=0\;s=0\;r=3 X-RBL-Hint: 209.132.180.67 rbl 9.3.list.dnswl.org.0 9.Y.list.dnswl.org.0 3.list.dnswl.org.0 18.rep.mailspike.net.0 X-GeoIP-Hint-1: 195.135.220.15 crc!Germany// dma!0 code!DE postcode! country!Germany region! city! code3!DEU area!0 latitude!51.0 longitude!9.0 lat1!51 lon1!9 lat90!90 lon90!0 lat2!52 lon2!10 lat45!45 lon45!0 lat3!51 lon3!9 lat30!60 lon30!0 lat5!50 lon5!10 lat18!54 lon18!18 lat6!54 lon6!12 lat15!45 lon15!15 lat9!54 lon9!9 lat10!50 lon10!10 X-DNS-Hint-1: 195.135.220.15 195 135 220 15 cantor2.suse.de cantor2 suse de X-ASN-Hint-1: Origin: AS29298 Net: 195.135.220.0/22 DE ripencc 2003_07_22 Desc: DE ripencc 2003_07_24 AS29298 SUSE LINUX GmbH,DE X-RBL-Hint: 195.135.220.15 rbl 9.3.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 9.Y.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 3.list.dnswl.org.1 suse.com http://dnswl.org/s?s=3640 X-RBL-Hint: 195.135.220.15 rbl 9.3.list.dnswl.org.1 9.Y.list.dnswl.org.1 3.list.dnswl.org.1 X-GeoIP-Hint-2: 195.135.220.254 crc!Germany// dma!0 code!DE postcode! country!Germany region! city! code3!DEU area!0 latitude!51.0 longitude!9.0 lat1!51 lon1!9 lat90!90 lon90!0 lat2!52 lon2!10 lat45!45 lon45!0 lat3!51 lon3!9 lat30!60 lon30!0 lat5!50 lon5!10 lat18!54 lon18!18 lat6!54 lon6!12 lat15!45 lon15!15 lat9!54 lon9!9 lat10!50 lon10!10 X-DNS-Hint-2: 195.135.220.254 195 135 220 254 X-RBL-Hint: 195.135.220.254 rbl none.2 X-Relay-Hint: ipv4 209.132.180.67 X-DSPAM-Bypass: relay=209.132.180.67 spam=492 ham=216192 bypass=62938 discard=0 since=2011-08-10 result=ham X-DSPAM-Result: Innocent X-DSPAM-Processed: Tue Jun 23 18:25:45 2015 X-DSPAM-Confidence: 0.9997 X-DSPAM-Improbability: 1 in 333510 chance of being spam X-DSPAM-Probability: 0.0000 X-DSPAM-Factors: 27, git+a, 0.00010, 7+#+#+static, 0.00010, file+changed, 0.00010, 100644+a, 0.00010, changed+#+insertions, 0.00010, 1+#+changed, 0.00010, 6+#+#+static, 0.00010, 6+#+#+static, 0.00010, index+#+#+#+a, 0.00010, c+#+#+#+changed, 0.00010, 100644, 0.00010, insertions+#+#+#+git, 0.00010, c+index, 0.00010, diff+git, 0.00010, insertions+#+deletions, 0.00010, deletions+diff, 0.00010, index+#+#+100644, 0.00010, deletions, 0.00010, 1+#+#+#+insertions, 0.00010, 1+file, 0.00010, struct+#+#+#+struct, 0.00010, c+#+#+#+100644, 0.00010, changed+#+#+#+deletions, 0.00010, int+#+#+#+struct, 0.00010, int+#+#+#+struct, 0.00010, Signed+#+by, 0.00010, 7+static, 0.00010 Content-Length: 7628 Lines: 265 ->readpage() does page_lock() before extent_lock(), we do the opposite in extent-same. We want to reverse the order in btrfs_extent_same() but it's not quite straightforward since the page locks are taken inside btrfs_cmp_data(). So I split btrfs_cmp_data() into 3 parts with a small context structure that is passed between them. The first, btrfs_cmp_data_prepare() gathers up the pages needed (taking page lock as required) and puts them on our context structure. At this point, we are safe to lock the extent range. Afterwards, we use btrfs_cmp_data() to do the data compare as usual and btrfs_cmp_data_free() to clean up our context. Signed-off-by: Mark Fasheh Reviewed-by: David Sterba --- fs/btrfs/ioctl.c | 148 +++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 117 insertions(+), 31 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 2deea1f..b899584 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -2755,14 +2755,11 @@ out: return ret; } -static struct page *extent_same_get_page(struct inode *inode, u64 off) +static struct page *extent_same_get_page(struct inode *inode, pgoff_t index) { struct page *page; - pgoff_t index; struct extent_io_tree *tree = &BTRFS_I(inode)->io_tree; - index = off >> PAGE_CACHE_SHIFT; - page = grab_cache_page(inode->i_mapping, index); if (!page) return NULL; @@ -2783,6 +2780,20 @@ static struct page *extent_same_get_page(struct inode *inode, u64 off) return page; } +static int gather_extent_pages(struct inode *inode, struct page **pages, + int num_pages, u64 off) +{ + int i; + pgoff_t index = off >> PAGE_CACHE_SHIFT; + + for (i = 0; i < num_pages; i++) { + pages[i] = extent_same_get_page(inode, index + i); + if (!pages[i]) + return -ENOMEM; + } + return 0; +} + static inline void lock_extent_range(struct inode *inode, u64 off, u64 len) { /* do any pending delalloc/csum calc on src, one way or @@ -2808,52 +2819,120 @@ static inline void lock_extent_range(struct inode *inode, u64 off, u64 len) } } -static void btrfs_double_unlock(struct inode *inode1, u64 loff1, - struct inode *inode2, u64 loff2, u64 len) +static void btrfs_double_inode_unlock(struct inode *inode1, struct inode *inode2) { - unlock_extent(&BTRFS_I(inode1)->io_tree, loff1, loff1 + len - 1); - unlock_extent(&BTRFS_I(inode2)->io_tree, loff2, loff2 + len - 1); - mutex_unlock(&inode1->i_mutex); mutex_unlock(&inode2->i_mutex); } -static void btrfs_double_lock(struct inode *inode1, u64 loff1, - struct inode *inode2, u64 loff2, u64 len) +static void btrfs_double_inode_lock(struct inode *inode1, struct inode *inode2) +{ + if (inode1 < inode2) + swap(inode1, inode2); + + mutex_lock_nested(&inode1->i_mutex, I_MUTEX_PARENT); + if (inode1 != inode2) + mutex_lock_nested(&inode2->i_mutex, I_MUTEX_CHILD); +} + +static void btrfs_double_extent_unlock(struct inode *inode1, u64 loff1, + struct inode *inode2, u64 loff2, u64 len) +{ + unlock_extent(&BTRFS_I(inode1)->io_tree, loff1, loff1 + len - 1); + unlock_extent(&BTRFS_I(inode2)->io_tree, loff2, loff2 + len - 1); +} + +static void btrfs_double_extent_lock(struct inode *inode1, u64 loff1, + struct inode *inode2, u64 loff2, u64 len) { if (inode1 < inode2) { swap(inode1, inode2); swap(loff1, loff2); } - - mutex_lock_nested(&inode1->i_mutex, I_MUTEX_PARENT); lock_extent_range(inode1, loff1, len); - if (inode1 != inode2) { - mutex_lock_nested(&inode2->i_mutex, I_MUTEX_CHILD); + if (inode1 != inode2) lock_extent_range(inode2, loff2, len); +} + +struct cmp_pages { + int num_pages; + struct page **src_pages; + struct page **dst_pages; +}; + +static void btrfs_cmp_data_free(struct cmp_pages *cmp) +{ + int i; + struct page *pg; + + for (i = 0; i < cmp->num_pages; i++) { + pg = cmp->src_pages[i]; + if (pg) + page_cache_release(pg); + pg = cmp->dst_pages[i]; + if (pg) + page_cache_release(pg); + } + kfree(cmp->src_pages); + kfree(cmp->dst_pages); +} + +static int btrfs_cmp_data_prepare(struct inode *src, u64 loff, + struct inode *dst, u64 dst_loff, + u64 len, struct cmp_pages *cmp) +{ + int ret; + int num_pages = PAGE_CACHE_ALIGN(len) >> PAGE_CACHE_SHIFT; + struct page **src_pgarr, **dst_pgarr; + + /* + * We must gather up all the pages before we initiate our + * extent locking. We use an array for the page pointers. Size + * of the array is bounded by len, which is in turn bounded by + * BTRFS_MAX_DEDUPE_LEN. + */ + src_pgarr = kzalloc(num_pages * sizeof(struct page *), GFP_NOFS); + dst_pgarr = kzalloc(num_pages * sizeof(struct page *), GFP_NOFS); + if (!src_pgarr || !dst_pgarr) { + kfree(src_pgarr); + kfree(dst_pgarr); + return -ENOMEM; } + cmp->num_pages = num_pages; + cmp->src_pages = src_pgarr; + cmp->dst_pages = dst_pgarr; + + ret = gather_extent_pages(src, cmp->src_pages, cmp->num_pages, loff); + if (ret) + goto out; + + ret = gather_extent_pages(dst, cmp->dst_pages, cmp->num_pages, dst_loff); + +out: + if (ret) + btrfs_cmp_data_free(cmp); + return 0; } static int btrfs_cmp_data(struct inode *src, u64 loff, struct inode *dst, - u64 dst_loff, u64 len) + u64 dst_loff, u64 len, struct cmp_pages *cmp) { int ret = 0; + int i; struct page *src_page, *dst_page; unsigned int cmp_len = PAGE_CACHE_SIZE; void *addr, *dst_addr; + i = 0; while (len) { if (len < PAGE_CACHE_SIZE) cmp_len = len; - src_page = extent_same_get_page(src, loff); - if (!src_page) - return -EINVAL; - dst_page = extent_same_get_page(dst, dst_loff); - if (!dst_page) { - page_cache_release(src_page); - return -EINVAL; - } + BUG_ON(i >= cmp->num_pages); + + src_page = cmp->src_pages[i]; + dst_page = cmp->dst_pages[i]; + addr = kmap_atomic(src_page); dst_addr = kmap_atomic(dst_page); @@ -2865,15 +2944,12 @@ static int btrfs_cmp_data(struct inode *src, u64 loff, struct inode *dst, kunmap_atomic(addr); kunmap_atomic(dst_addr); - page_cache_release(src_page); - page_cache_release(dst_page); if (ret) break; - loff += cmp_len; - dst_loff += cmp_len; len -= cmp_len; + i++; } return ret; @@ -2904,6 +2980,7 @@ static int btrfs_extent_same(struct inode *src, u64 loff, u64 olen, { int ret; u64 len = olen; + struct cmp_pages cmp; /* * btrfs_clone() can't handle extents in the same file @@ -2916,7 +2993,7 @@ static int btrfs_extent_same(struct inode *src, u64 loff, u64 olen, if (len == 0) return 0; - btrfs_double_lock(src, loff, dst, dst_loff, len); + btrfs_double_inode_lock(src, dst); ret = extent_same_check_offsets(src, loff, &len, olen); if (ret) @@ -2933,13 +3010,22 @@ static int btrfs_extent_same(struct inode *src, u64 loff, u64 olen, goto out_unlock; } + ret = btrfs_cmp_data_prepare(src, loff, dst, dst_loff, olen, &cmp); + if (ret) + goto out_unlock; + + btrfs_double_extent_lock(src, loff, dst, dst_loff, len); + /* pass original length for comparison so we stay within i_size */ - ret = btrfs_cmp_data(src, loff, dst, dst_loff, olen); + ret = btrfs_cmp_data(src, loff, dst, dst_loff, olen, &cmp); if (ret == 0) ret = btrfs_clone(src, dst, loff, olen, len, dst_loff); + btrfs_double_extent_unlock(src, loff, dst, dst_loff, len); + + btrfs_cmp_data_free(&cmp); out_unlock: - btrfs_double_unlock(src, loff, dst, dst_loff, len); + btrfs_double_inode_unlock(src, dst); return ret; } -- 2.1.2 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html