KEMBAR78
Implement local lock cache, support querying it by sinbad · Pull Request #1760 · git-lfs/git-lfs · GitHub
Skip to content

Conversation

sinbad
Copy link
Contributor

@sinbad sinbad commented Dec 12, 2016

The next incremental rewrite of the locking work; this time we're using our key value store to locally cache locks for the current user for more efficient access. This will be used in the next PR to provide efficient support for marking files read-only on checkout if not locked.

Also adds some minor API improvements such as returning a full Lock structure on acquiring a lock rather than just an Id.

@sinbad
Copy link
Contributor Author

sinbad commented Dec 12, 2016

I think with the merging of this PR, locking-master can be merged into master because it's a stable mid-stage. There will then be a few further staging posts between here and when the locking is completely finished.

Copy link
Contributor

@ttaylorr ttaylorr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, I left some thoughts about a potential further abstraction you could make along with some nit-picks.

// path must be relative to the root of the repository
// Returns the lock id if successful, or an error
func (c *Client) LockFile(path string) (id string, e error) {
func (c *Client) LockFile(path string) (Lock, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think returning a *Lock would be appropriate here, since this method doesn't guarantee that locking a file will succeed all of the time, and the zero-value Lock{} doesn't make sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I remember this was deliberately by value now. Returning a non-pointer is important because I don't want to allow the client to modify the original lock, because it's cached and therefore they could corrupt the cache by doing that. This outweighs the zero value Lock concern IMO. Yes I could return a pointer and copy it but it's unnecessary complexity. A zero value in the event of an error is idiomatic anyway (nil is just a zero value pointer after all).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't mind implementing a (l *Lock) Copy() *Lock function, but I don't feel strongly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That depends on callers doing the right thing (not modifying return values from the locking client. I think cache corruption is a good enough reason to return a value.

locking/locks.go Outdated
// that this lock was created. For most server implementations, this
// should be set to the instant at which the lock was initially
// received.
// LockedAt tells you when this lock was acquired.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/tells you/is the/

// If localOnly = true, don't query the server & report only own local locks
func (c *Client) SearchLocks(filter map[string]string, limit int, localOnly bool) (locks []Lock, err error) {

if localOnly {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😻

locks := make([]Lock, 0, len(cachedlocks))
for _, l := range cachedlocks {
// Manually filter by Path/Id
if (filterByPath && path != l.Path) ||
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I follow this logic, but I think we can make it clearer:

for _, l := range cachedLocks {
        matchesPath := path == l.Path
        matchesId := id == l.Id

        if (filterByPath && matchesPath) || (filterById && matchesId) {
                // ...
        }
}

This does add a level of indentation to the body of the loop, but I think the extra clarity is worth it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hate to be the optimisation nitpicker here but doing it my way avoids comparison operations in the case of !filterByPath or !filterById whereas your way requires the comparison all the time, so I prefer my way 😄 I think it's clear enough.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that the original is clear enough.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

locking/locks.go Outdated
// that won't be in a path)
const idKeyPrefix = "*id*://"

func (c *Client) encodeIdKey(id string) string {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if (c *Client) is the right receiver for this method. What do you think about wrapping tools/kv in this package with functions that understand *Lock and etc, and putting this method there?

package locking

const (
       idPrefix string = "*id*://"
)

type LockCache struct {
        kv *kv.Store
}

func (c *LockCache) CacheLock(l Lock) error {
        // ...
}

func (c *LockCache) encodeId(id string) string {
       if !c.isId(id) {
               return idPrefix+id
        }

       return id
}

func (c *LockCache) isId(key string) bool {
       return strings.HasPrefix(key, idPrefix)
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this approach a lot. I think it's a nice, clear line of abstraction that we can draw. The *locking.Client would own a *LockCache underneath and delegate into the cache's public methods from there.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's a good shout.

config.LocalGitStorageDir = oldStore
}()

client, err := NewClient(config.NewFrom(config.Values{}))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's another place where I think the approach I proposed above would yield a win. This test is testing the behavior of caching a lock, but creates a locking client to do so. Moving that behavior down into a more isolated part of the system would mean that this test can be closer to it's subject.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, it made it better :)

locking/locks.go Outdated
func (c *Client) cacheUnlockByPath(filePath string) error {
ilock := c.cache.Get(filePath)
if ilock != nil {
lock := ilock.(*Lock)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that this is likely behavior, but I think that using the two-tuple variant of this cast would be safer in that it'd prevent an unlikely panic:

func (c *Client) cacheUnlockByPath(filePath string) error {
        ilock := c.cache.Get(filePath)
        if ilock == nil { // <- needed to prevent `(*Lock)(nil)`
                return nil
        }
        if l, ok := ilock.(*Lock); ok {
                // ...
        }

        return nil
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I've just collapsed this into:

if lock, ok := ilock.(*Lock); ok && lock != nil {

Since the actual casting of nil is safe anyway so long as we check.

locking/locks.go Outdated
c.cache.Remove(idkey)
c.cache.Remove(lock.Path)
}
return nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should return an error if we couldn't find the lock? I think a sentinel value here would be fine:

package locking

var ErrMissingLock error = errors.New("locking: could not find lock")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I always think it's better to fail silently when requesting to remove a non-existent item from any collection; this is idiomatic with core collections and what I would expect from an API.

locking/locks.go Outdated
return nil
func (c *Client) cachedLocks() []Lock {
var locks []Lock
c.cache.Visit(func(key string, val interface{}) bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This API is awesome 👍

Copy link
Contributor

@technoweenie technoweenie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some naming concerns, but that's it. If you feel strongly about your names, go ahead and merge. Or we can discuss it in the locking-master -> master PR (our last chance to finalize exported names) :)

locking/cache.go Outdated
}

// Cache a successful lock for faster local lookup later
func (c *LockCache) CacheLock(l Lock) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having functions with a Cache prefix is redundant. Especially in the *Client usage: c.cache.CacheUnlockById(id). What do you think about method names like this:

  • CacheLock -> Set
  • CacheUnlockByPath -> DeleteByPath
  • CacheUnlockById -> DeleteById
  • CachedLocks -> GetAll or just All

Copy link
Contributor Author

@sinbad sinbad Dec 14, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point; to keep it distinct from kv I'll make it Add / RemoveByBlah / Locks.

locks := make([]Lock, 0, len(cachedlocks))
for _, l := range cachedlocks {
// Manually filter by Path/Id
if (filterByPath && path != l.Path) ||
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that the original is clear enough.

locking/locks.go Outdated
}
// Fetch locked files for the current committer and cache them locally
// This can be used to sync up locked files when moving machines
func (c *Client) fetchLocksToCache() error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think something like refreshLocksCache() would be a better name.

Copy link
Contributor

@ttaylorr ttaylorr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sinbad - thanks for addressing my comments 😄. I'm with @technoweenie on this one: everything looks good with some naming changes, unless you feel strongly.

@sinbad sinbad merged commit 086779a into locking-master Dec 14, 2016
@sinbad sinbad deleted the locking-wip/cache branch December 14, 2016 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants