> First, you’ll need to find the data that needs to be deleted.<p>Microservices do not get you out of having to have an information architecture. They add more friction if you don't have one, but it's entirely possible to have an unspoken/undocumented information architecture that mostly works.<p>If you don't have a System of Record for data, you for sure aren't going to be able to find it. Similar problem with no Source of Truth. For some business models you will have both and they will be separate (especially with 3rd party data).<p>You still have the problem of logs, but at least the problem is tenable. Without any of this it's just chaos and who knows where the data went or really even where it came from?
One should really consider DevSecOps when dealing with a large infrastructure involving dozens or hundreds of micro services. Security as Code by design should be implemented as soon as possible. In my current gig - the following 3 points are key:<p>- 24x7 Proactive Security Monitoring<p>- Shared Threat Intelligence<p>- Compliance Operations<p>I would really advise anyone interested in this subject to check out DevSecOps manifesto [1].<p>Also - if you are running a k8s cluster for all your micro-services, there are several guides online as to best practices (see [2] for example).<p>----------<p>[1] <a href="https://www.devsecops.org/" rel="nofollow">https://www.devsecops.org/</a><p>[2] <a href="https://dev.to/petermbenjamin/kubernetes-security-best-practices-hlk" rel="nofollow">https://dev.to/petermbenjamin/kubernetes-security-best-pract...</a>
This is relevant for enterprises with legacy systems, too, like shops that have multiple interfaces that extract, transform, and load data across point-of-sale systems, warehouse fulfillment systems, and data warehouses.
This gives me nightmares about having to retrofit GDPR requirements into a complicated system with many data stores and applications. Not only the difficulty of ensuring data is deleted but tracking data lineage so you can delete derived data too. Fun times!
This will help us <a href="https://bitmovin.com/finding-memory-leaks-java-p2/" rel="nofollow">https://bitmovin.com/finding-memory-leaks-java-p2/</a>
My company recently decided to switch over to microservices. It was a long an arduous process but we chose not to compromise.<p>Basically for the greatest amount of modularity, we divided all 400 functions in our monolithic application into 400 individual servers because obviously functions aren't modularizing everything enough. You really need to put more and more wrappers around all of your functions. First put a framework around your function, than put an http api layer around it, then wrap a server app around it, then put an entire container around it and boom! More wrappers == Less technical debt. To illustrate how this works see example below:<p><pre><code> wrapper(
wrapper(
wrapper(
wrapper(
f(x)
)
)
)
)
</code></pre>
See that? Obviously for every additional wrapper you add around your original function your technical debt becomes less. This is why it makes sense to make to not just use functions to modularize your data but to wrap all your functions in containers and then put those containers in containers.<p>Now all 400 of our engineers each as an individual manages one entire function within one entire container. It's amazing, they don't have to think about two things anymore, they can just concentrate on one thing.<p>While I'm not sure what has improved yet, everything feels better. Our company is following industry trends and buzzwords. Technical debt actually went up but that's just our fault for building it wrong.<p>Some engineer asked me why couldn't we just load balance our original monolith and scale it horizontally. I fired that engineer.<p>Another engineer came to me and told me that for some function:<p><pre><code> func some_func(a: int, b: int) -> int: {
return a + b
}
</code></pre>
It's probably better to do 1. instead of 2.<p><pre><code> 1. Call the function: some_func(2,3)
2. Make a request: request.get("http://www.pointlessapi.com/api/morecomplexity/someextratechnicaldebt/randomletters/asdfskeidk/some_func?a=2&a=3")
and parse the json:
{
"metadata": {
"date": "1/1/2020"
"request_id": "123323432",
"other_pointless_crap": ...
"more_usless_info": ...
"function_name (why?)": "some_func"
}
"actual_data": 5
}
</code></pre>
I fired that engineer too. Obviously 2. is better than 1. Also why am I using JSON? Not enough wrappers! You have to wrap that http call in additional wrappers like GraphQL or GRPC!!! (See wrapper logic above).<p>Have you guys heard of a new trend called Sololithic architecture? Basically the new philosophy states that all of 400 of our microservices should be placed in 400 containers and run under a cluster of 399 computers under kubernetes!<p>I may not understand where technical debt comes from and I also may not understand how all these architectures will fix the problem of technical debt forever... I know that industry trends and buzzwords are more intelligent than me and monoliths are bad bad bad and obviously the source of all technical debt! Just cut everything into little pieces and technical debt becomes ZERO.<p>Right? The technical definition of Technical debt is not enough cutting of your logic into tiny pieces and not enough wrappers around your modules so all you need to do is cut everything up and put wrappers around it and problem solved! Makes sense!