I don't remember the exact project or where I read it but one project added delay (using sleep function) to depreatected methods in order to discourage people using them. After every release, they increased the sleep time so that at one point it became impossible to use it without noticing it. Although I don't know if it is possible to do this for new code trying to use deprecated function and have old code use non-delayed one, I think it was a nice trick.
Here, a “shitlist” tracks existing uses of deprecated behavior within a large code base. It prevents new uses of deprecated behavior without disrupting legacy code. This is a temporary measure used to facilitate refactoring large code bases.
This is a good example of "Organization as Code", where you are trying to influence a non-trivial change (often "cultural") by code.<p>Frequently it's better to change behaviour this way than say, holding meetings and presentations.<p>A typical example is "we want our developer to write more tests" but there are few existing examples of test code to look at and when you write a test it takes forever to run, so you fix this by fixing the underlaying issues (which is bad or non-existing code), rather than making developers attend TDD presentations and just talking about a "test culture" for example.
If you don't want people to use deprecated code, it's essential to document what the replacement should be. In JavaWorld, this can be done through Javadoc. Just tagging a method or class with @Deprecated and leaving it there is the sort of thing that makes me want to hurt people.
It's better to use a Shiterator, which can yield pieces of shit on demand, lazily enumerate them just in time, and generate infinite streams of shit, instead of allocating all your shit up front, and having your shit together in one place.
Brilliant. It's a really good compliment to strangling certain parts of a large code base.<p>As always the key is getting the information to the right people at the right time, and making it more difficult to make the wrong choices then the right choices.
This is an excellent way of using your (hopefully) already existing testing and CI infrastructure to change a social problem into a technical problem. As we all know, technical problems are by far the easier of the two. :)<p>We used this technique to great effect at my previous day job, though under the more neutral name of "ratchet".
The Ruby-based tooling has been released as open source: <a href="https://github.com/Shopify/deprecation_toolkit" rel="nofollow">https://github.com/Shopify/deprecation_toolkit</a>
This is accomplished at Google using the build system. Your visibility declaration tells the build system what's allowed to depend on a piece of code.<p>When deprecating old code, you can shut that door with a whitelist and gradually force consumers to migrate.<p>I really like how code is managed at Google.
A similar kind of automated mechanism is required in distributed systems that allow for rolling upgrades. New functionality in upgraded nodes can't break not-yet-upgraded nodes and legacy behavior in not-yet-upgraded nodes has to be tolerated in upgraded nodes but only until the entire system is upgraded and then it is prohibited. Doing this wrong results in some really hard to fix production states.
On a somewhat related note: One of the biggest things I've noticed is that simpler pieces of software are less likely to be deprecated, whereas complex codebases quickly become legacy and abandoned...
In a similar vein but on a smaller scale we have a number of tests for deprecated behaviors in our application code, and whitelist the existing code based on serializing the file name, method name, and the ordered list of parsed token <i>types</i> of the method.<p>That way minor alterations like changing a string or numeric value don't remove the method from the whitelist, but alterations to the logic of the given method require you to fix the issue while you're in there poking around already to pass CI.
For the Java Developers here, have a look at archunit. We use this to discourage certain behaviour but you can also use it to test and whitelist the usage of shitlist items => <a href="https://www.archunit.org/" rel="nofollow">https://www.archunit.org/</a>
I love his specific examples.<p>> Make sure that a certain datastore is only read from in a certain context ... Ensure fallbacks for all uses of a secondary data-store ... joins between tables that have no business being joined<p>I wish linters / typesystems were extensible enough to do this kind of domain-specific checking. There's a new generation of static analysis that's much more focused on architecture or business rules and is less about code-in-the-small concerns like class methods or operator compatibility.
That's a pretty good idea.<p>I think a key to doing large, multi-developer projects, is true modular design, with opaque APIs, and each module with its own project identity and lifecycle.<p>Not a particularly popular stance, as it means a lot more overhead in each project.<p>It does reduce the need for shitlists, though.
Even in a single-team project, I've ran into the problem that I'm refactoring some old code while others keep adding to it. Some way to have to cause failing tests sounds like a great idea.
Another cool way you could do this: Ruby methods can ask for their caller_locations. They work like this:<p><pre><code> # Source
$ cat example_for_hn.rb
def foo
bar
end
def bar
baz
end
def baz
(c1, c2) = caller_locations.first(2)
puts "Parent caller: '#{c1.label}' in '#{c1.path}'"
puts "Grandparent caller: '#{c2.label}' in '#{c2.path}'"
end
foo
# Demo
$ ruby example_for_hn.rb
Parent caller: 'bar' in 'example_for_hn.rb'
Grandparent caller: 'foo' in 'example_for_hn.rb'
</code></pre>
So, you could define a method decorating class method like so:<p><pre><code> module Shitlist
def shitlist(method_name, whitelist)
original_method = instance_method(method_name)
undef_method(method_name)
define_method(method_name) do |*args, &block|
call = caller_locations.first
passes_whitelist = whitelist.any? do |label, file_pattern|
call.label == label && call.absolute_path.end_with?(file_pattern)
end
unless passes_whitelist
fail "Shitlisted method! Permitted callers: #{whitelist}"
end
original_method.bind(self).call(*args, &block)
end
end
end
</code></pre>
and then extend classes with it to use the decorator:<p><pre><code> class Example
extend Shitlist
def not_on_shitlist
qux
end
def baz
qux
end
def qux
puts 'Only some methods can call me :)'
end
shitlist :qux, 'baz' => 'shitlist.rb'
end
</code></pre>
If I run this example (full source: <a href="https://git.io/JLOdV" rel="nofollow">https://git.io/JLOdV</a>), the non-whitelisted caller throws an error:<p><pre><code> $ ruby shitlist.rb
Only some methods can call me :)
Traceback (most recent call last):
2: from shitlist.rb:44:in `<main>'
1: from shitlist.rb:25:in `not_on_shitlist'
shitlist.rb:13:in `block in shitlist': Shitlisted method! Permitted callers: {"baz"=>"shitlist.rb"} (RuntimeError)
</code></pre>
---<p>Of course, you might not want this hijacked method with tracing inside something performance critical. You could always configure the implementation to be a no-op in production.
I'm more curious about how things like shitlists are implemented in different languages.<p>Most of my experience is C#, where calling deprecated code triggers a warning.<p>Considering that C# warnings will fail in CI, how would someone do a C# shitlist? Would it require some kind of #pragma, that would stick out like a sore thumb in a code review
To an extent this seems to be related to more general automated detection of refactor targets/"code smells" (ironically) for which there exist tools already (e.g. linters indicated in the article) - I like this simple "list" approach though.
Love the concept, hate the name. To me this seems like the recycle bin concept. Allow the code to exist temporarily, but encourage permanent deletion. Perhaps a better name could be a "screamlist" or "needstochange" list.
I kept a personal shitlist in one project but it was rather a collection of "worst practices" - things I vowed never to do.<p>It was personal because the idea didn't get a lot of attention when I brought it up.
We use Bazel’s visibility settings this way. More often for controlled beta rollouts than for deprecation, but same principle. You can make it a compile time error to import a package that isn’t for you.
I love that C# supports deprecation as a language feature and allows to mark an obsolete method as only usable by other deprecated methods, which is the same as the shitlist approach here in the end
Shitlist driven migration/refactor, maybe. If it's <i>development</i> it means you're always deprecating something and that's a significant organizational smell.
In case you wonder what shitlist means:<p><i>> “shitlists”: a whitelist of deprecated behaviour. Existing deprecated behaviour is OK and whitelisted.</i>
The article shows ways to prevent new dependencies on your deprecated library/service. This is half of the problem. The other half is efficiently removing the existing dependencies.<p>Your team could change the other teams' code to remove the dependencies. This is usually inefficient. You will waste time learning their code. They may drag out the code review process for weeks or months.<p>Some teams may refuse to accept your changes and use your service deprecation as political capital to demand more headcount. They may even lie to their managers and claim that the dependency deprecation is justification for a rewrite that they want to do.<p>There is a technical solution that can help with this social problem: AUTOMATICALLY EXPIRING DEPENDENCY APPROVALS. Configure your library to allow only existing systems to use it, and make them all break on a certain date. Then, instead of forcing the other team to move, they have to move or their build breaks. And if they want to delay turndown they must convince you to change your code. Without automatic expiration, they can delay turndown by simply ignoring you.<p>Some teams may wait for the dependency expiration, lie saying they didn't know about the turndown, and then demand that you delay the turndown and give them more time. You can work around this with a two-phased turndown. First create a new version of the library that allows only existing clients. Give the library a hideous name so code owners will want to remove it. Example: Deprecated_LiBrArY_YoUr_BuIlD_WiLL_BREaK_oN_20200601_LOL_Were_sERIUS_YOLO_exxtensuns_COme_frun_SVP_DaniELs_OnLY. Then set the existing library to expire in a week and email all users. They can easily switch to the new hideously-named library and in the process acknowledge that they know that their build will break at the specified date.<p>TLDR: Use expiring white lists so you won't get ignored. Rename your deprecated library to something hideous to motivate code owners to migrate away from it.
Much better to annotate those functions with @deprecated and allow running in "deprecated allow mode" and a "deprecated fail mode"...<p>Never use profanity in your code, nor in your comments. Lets keep things professional.
One can't always introduce a multitude of shitlists into different places in the code. This can impact performance and readability; and perhaps even more importantly - if the language used is not high-level with reflection capabilities, you will be counting on callers' participation in calling the semi-deprecated API with truthful "usage key" for looking up on the shitlist. Who's to say nobody reuses authroized keys for expediency?
I will probably remember Sirupsen forever for his move of breaking Golang projects by renaming his github username. This led to go modules freaking out because different project used different name casing for the dependency. He has a popular logging library and go modules are angry when you rename github usernames..