The article is discussing documentation for the AWS Flow Framework specifically. The Flow Framework is a Java framework built on top of the SWF API, and it provides a completely different programming model than the SWF API.
The C# example being discussed, as well as the Java example at the end, are examples of using that SWF API directly. The SWF API is indeed simpler for trivial examples. Flow is a power tool that handles complex workflows better than any alternative I've seen, but the framework itself is complex and incurs cost to set up and use. The documentation could do a better job of explaining this, and of providing Java API examples.
The Flow framework provides something that's a mix of Java code and a domain-specific language for building SWF applications that's expressed as Java code. (For an analogy, consider EasyMock.) It's hard to explain Flow concisely, but if I had to try I'd say, "You write code that looks like it's procedural and runs on one machine, and Flow converts that into a distributed workflow running across of fleet of machines, all of which may be stateless". Flow threads the state into and out of SWF for you, making it possible to express distributed workflows at a higher level of abstraction. Some examples are in the AWS Flow Framework Recipes: https://aws.amazon.com/code/2535278400103493
To achieve this, Flow uses fancy code weaving & AOP techniques. This makes it more complicated to set up, develop, and test. Flow pays off however once your workflow is more complex than a simple linear workflow. You could, for example, process data with a distributed map/reduce pattern in a few lines of code in Flow. (Source: built production systems on SWF with and without Flow)
It's unfortunate that it's the first Java example provided for anything to do with SWF. I got dumped into that from the "what is SWF" page. There's no simple Java SWF example like there is for C#. The docs dump you straight into the Flow Framework.
I'd argue based on Occams Razor that a more simplistic start is what most people want. Unfortunately to get that you have to look at the documentation of another language.
The counter argument is Flow IS what you want. Why poll for decision tasks and try to schedule the work execution between different servers when SWF Flow can do it for you?
All the things you found irritating are a one-time setup, and once you are chugging along you can write incredibly quickly powerful distributed systems without worrying about scaling or scheduling of any kind.
But I agree the documentation could do a better job of SELLING you on that point up-front.
Truth is, I don't know of any other framework or service that allows you to simply write java code like this:
A = getA();
B = getB();
C = getC(A, B);
D = getD(C);
E = getE(B);
Assuming all these getCalls are complex, time-consuming, and stateful, Flow automatically sets it up so that at the start getA() and getB() start executing in parallel (and on different servers!), getE() i immediately started once B is realized, getC() is immediately executed once both A and B are both realized, and so on.
For complex workflow with undetermined execution time where the whole thing may take days to execute, the fact that you can just tweak a simple Java file to update your business logic, and add some servers to your fleet whenever you want to improve throughput is MAGICAL.
Not all problems in software are simple to abstract away. I think Flow does the best it can do
Yes. It's more complicated than it needs to be, but to jump from this to "aws java doc sucks"? Most AWS doc (java or something else is pretty straightforward).
And it's not so much about AWS quality of Java documentation.
It's just about "why the heck does everything Java have to be so damn heavyweight and so damn hard".
Everything in the Java ecosystem is hard. I generally don't care so much about what language something is built in but I do try really hard not to use major applications written in Java. Life is too short to spend my days fighting through thickets of XML tags to try to get something configured, or installing something that seems to be constructed from hundreds or thousands of files.
Use a different language and some things aren't so hard any more.
I will say that the exception is the JetBrains IDE that I use most of the day is built with Java and somehow they have managed to shield me from ever needing to even know that.
It really isn't. I request that you qualify your comments to the subset of the "Java ecosystem" that you actually have experience with. Because "thickets of XML tags" really has nothing to do with Java, though it might have to do with certain massive frameworks built on top of Java.
I'm curious, what parts of the Java ecosystem (other than simple stuff, or old stuff, or non-Java languages on the JVM) don't involve thickets of XML tags? I have so little Java experience that it's fallen off my résumé, so I'm sure I'm uninformed and would like to be more informed. But everything I've seen of actual, production Java development today heavily uses XML somehow, from Ant or Maven build files, to web.xml, to Spring dependency injection, to Android, to even Java Web Start.
Is there any serious part of the Java (Java, not Scala or Clojure) ecosystem that doesn't involve XML at every corner?
Well, for starters, Spring basically has eliminated XML configuration with Spring-Boot. Add a couple annotations and you're done (as in, fully done, no web.xml's, no application-contexts). (Granted, I don't really like this, from an engineering point of view, but whatever).
Also, I really wouldn't complain about Maven, as it is hands down the best dependency manager available from any language, bar maybe gem/rails (node isn't even close, since you need a c++ compiler to actually get quite a few packages). There are also enough plugins for every IDE out there to handle Maven POM's.
But if that wasn't enough, there's now Gradle. You can literally have an entire Spring/Java/Grade project that doesn't use ANY XML, but purely Annotations/DSLs.
Secondly, there's a major selection bias at play. In your own words "everything I've seen of...", sorry but your point was lost right there.
Yes. I've seen (and written) plenty of actual production Java code that doesn't use any XML configuration at all.
There are a lot of issues with Java, but XML isn't INHERENT to it.
Annotations are IMHO just another and admittedly shorter representation of the same problem.
You still have to learn them by heart for each lib, you don't know when they are executed and you cannot reason about them (as you could with source code).
Pretty much the same problems as XML if you ask me.
If by the same problem you mean that you have to understand a library in order to use it properly, I guess so, but I don't see how that's unique to Java.
I don't see any difference between having to understand annotations and having to understand libstdc++ for example, or having to learn about any particular ruby gem or node package.
Every language has to deal with configuration files. Complaining because a bunch of Java developers used XML in their projects seems trite.
I think XML actually has a lot of strengths (having well-defined-types and structure for one). If the complaint is that it's worse than JSON, well...I've seen way worse than XML (ASN.1 for instance), so idk...
I'd put XML and annotations in the same class of problem because they are (mostly) not checked by the type system. For instance, you can have a REST controller that simply does not "work" because you're missing an annotation. You can easily debug this if is a place in your code where the controllers are registered, but with annotations the interactions between your code and the framework (or whatever requires annotation) are not obvious by just looking at the code.
Not really. Annotations are included with the code, unlike XML. They are much easier to reason about due to the immediate proximity. How do you feel about Python decorators? It's the same thing.
Thanks for the reply, good to know! I hadn't heard of Spring Boot and had forgotten about Gradle.
In case I wasn't clear - I wasn't trying to have any sort of "point," merely asking for the sake of learning what the Java world looks like. I think my experience with Java is pretty average for a non-Java developer, though (I took one class in college, and worked a few years on the client side at an enterprise software startup, where the server was J2EE, and had no particular influence on what stacks were used in both contexts). I certainly am not complaining about Maven, just relating the fact that it involves lots of XML.
I totally get that XML is intimidating, it certainly was for me for a long time, but eh...it solves more problems than it creates (at least in my professional opinion).
But... I'll be the first to admit that Java isn't the perfect tool for every problem, and I certainly feel that as engineers, we should use the proper tool to solve the problem.
> as it is hands down the best dependency manager available from any language
Why? You don't back that statement up with anything. What makes it better than `pip`?
> node isn't even close, since you need a c++ compiler to actually get quite a few packages
Well if the packages contain C/C++ modules and don't distribute binaries... (which a lot do). In Python land packages with a C++ module normally have a pure-python fallback anyway.
> In Python land packages with a C++ module normally have a pure-python fallback anyway.
Completely not true in my experience. Running pip install and finding out a dependency is native code is the worst feeling, because now I need to replicate a dev environment for building the damn thing, on windows I don't even bother, on linux I'm left hunting down the right linux packages, which if someone bothered to specify was only specified for one of the distros.
> Why? You don't back that statement up with anything. What makes it better than `pip`?
I'm not super familiar with the python ecosystem, but my biggest gripe is above (randomly, dependencies will fail to install). My second gripe with python is that so many things need virtualenv.
In this regard I've had a lot less pain with npm (but I've also done far fewer things in node), and I've never had an issue using a library in the java ecosystem, I just plug the appropriate deps into my build system, and it works, no installation required.
Beyond "can I use the packages easily", Maven also supports artifactory so that we can have org-wide libraries rather than needing to rely on git to avoid duplicating files everywhere. Beyond that Maven is also the Grunt of the Java world; it compiles your code, it invokes JUnit, it runs your fancy code rewriting modules, it does your jar/war packaging, and runs all the other modules you want.
Maven has it's own issues sometimes, but having too much XML is so far down the list of things I care about, but Gradle is nice too and the syntax is much more terse and I try to use it for my own non-work projects.
In contrast to the general "install a pile of things not in your dependency manager" approach of pip/npm, I just setup a java project to use webkit (with no prior experience trying to do this in java at all) in 5 minutes by adding 2 entries to a pom.xml file as described on https://github.com/ui4j/ui4j/
Recently moving from Python to Java. `pip` doesn't even come close to Maven. Dependency management used to be something I needed to manage much more closely in Python. With Maven, it just works.
I'm just curious as to why. You specify a list of requirements and their versions, and those get pulled down from somewhere and installed locally. What makes Mavens process far superior to pip (or npm, or any other program)?
Because with JVM languages, and their "horrid" backwards compatibility (that everyone always loves to bash), I can declare dependencies on libraries written in Java 1.1, and it will JUST WORK.
From what I understand, there's a fairly large chasm between Python 2/3 (such that a lot of people use external libraries to deal with coding cross-compatibile code, and apparently get wrong some-times).
Well, I don't have that much experience with Python, but by your own admission:
packages with a C++ module normally...
Sorry, but 'normally' doesn't cut it for me (or the organizations that I work/worked for).
I can use ANY maven dependency, regardless of what JVM language it was built with, what language level (java 1.1 - 1.8) it was built with, pretty much anything.
I do Java projects for more than 15 years. I know the times when XML-based configuration was the standard. Now, I am working on the MVP of my startup, with Dropwizard. The XML days feel like stories from a fairy tale.
But I have to say that XML is often blamed by people who think that complexity would magically vanish. Complexity does not vanish. It can only be shifted around. And it often requires lots of iterations to find an approach that is good enough for everyone. Back in the days, if tooling was right, I did not care about the amount of XML.
As a Java developer doing mostly enterprisey stuff, I haven't used XML in years. Yes, web.xml is maybe the only piece of XML I have to deal with in certain cases.
Lots of Java packages don't use XML. Just on the web app servers: Play, Vertx, Spring MVC, Dropwizard, and others don't use XML. The new crop of micro framework stirred clear of XML: Spark Framework, Pippo, Rapidoid, Blade, and Jodd.
There is ONE exception, when I decided to mod the game Starsector, setting up the project, and compiling and running while using IntelliJ is far easier than any other Java project I ever had, also no XML anywhere, it so wonderful it ALMOST made me like Java.
I'm a java developer. Why should I look at a eclipse project file? (Ok, as I'm using Idea - why look at an Idea project file?) Those are the internal storage systems of the IDE and I don't care what they use. And I won't go around patching the IDE's config files manually ever.
Why because the project you need to swipe some stuff from has no build system other than the wretched eclipse project file. Put it this why, you look at the project file, it's a tell how horrid everything is with the Java ecosystem that someone thinks that is a good idea.
This is precisely about the quality of the documentation and the provided example. A simple C# example is included and works in Java with minor modifications. It's at the end of the article.
I'm not really seeing it. This example is egregious and that's what the author is upset about. If it had, I dunno, just a Maven pom of some sort it would be in the same ballpark of conceptual overhead as npm/grunt/gulp/grumble/glorkle/grlarlarlgh/rvr/gem/bundler/virtualenv/easy_install/pip/hip/hooray/etc.
You're affirming the consequent. The truth is that "it's the enterprisey way to use Java", which is what leads to examples like this. The converse is not true, which is that it's the Java way to be enterprisey.
It would be foolish to explicitly pay for lines of code, but there are lots of situations where that happens implicitly.
Most programmers get paid by the hour, and consistently produce about the same number of lines of code per hour.
Most large contracts are billed according to the perceived magnitude of the task. The value to the client is more important than the cost to the contractor, but the cost is still a factor. And for a contractor who has already decided which programmers will work with which tools, that cost is directly proportional to lines of code.
> Life is too short to spend my days fighting through thickets of XML tags to try to get something configured, or installing something that seems to be constructed from hundreds or thousands of files.
> I will say that the exception is the JetBrains IDE
As someone who wrote a few patches for Idea, I can say that its one of the most bloated and complex java applications in existence. I do not mean it in a negative way, it just solves complex problem. And yes it uses XML for configuration.
As someone who writes Android apps everyday and has no prior experience with the Java ecosystem, seems fine to me. I put my deps in gradle and let Android Studio sync, then I just.. code. The IDE does most of the work.
That "hello world" project has now been incubated as an Apache Open Source Project.
You can download and install Apache Hello World for Java Enterprise Edition on an enterprise scale (assuming of course that you have sufficient bandwidth remaining for this month).
Support is available for "Apache Hello World Enterprise Edition" from Tata Consulting Services, HP Consulting Service and if that's not expensive enough for you, Price Waterhouse are now offering dedicated fly-in-from-overseas teams who have the skills and experience to roll out "Enterprise Hello World" in record implementation time of generally less than 6 months, not including customisations to your environment, which will be built by university graduates at $1,200 per day.
Business Analysts are ready to write sophisticated UML diagrams to map out how to integrate Apache Hello World in carefully considered ways that match your business processes.
New Relic will provide application instrumentation for your Apache Hello World implementation to ensure if (when?) it goes down you'll be able to rapidly pinpoint the general 50,000 code lines area that the fault might lie.
SWF is a developer tar baby. Don't touch. It's incredibly complex for what little it does. At the service side, a simple queue server would replace it. Everything about the 'workflow' is implemented on the client using whatever latest java abomination like AOP backed annotations.
Author here. I thought that too at first. It's actually straightforward though. I now have a video transcoding workflow that works well.
It's not SWF that's the problem. It's actually really straightforward to use. AOP is not required, annotations are not required and those design patterns are not required. It's just the really messed up documentation that makes you think this.
Check the bottom of the above article. You just take the C# sample and copy and paste it onto Java. Change the syntax to Java. It gets a bit more verbose but is still trivial to grok.
"Enterprise" Java (app servers, etc.) sucks but there are light weight alternatives (such as Dropwizard) that don't have the problems of 10+ years ago.
There's no XML garbage and there's minimal boilerplate.
And, as a bonus, with Java you get real threading, don't spend all day working around single threaded interpreter GILs or dealing with call back hell. You can actually use all the cores on those CPUs, in a single process if you want!
And then you find out pjlegato switched to Elixir, where he gets lightweight libraries by default, real threading, no GILs, no callback hell, uses all the cores trivially, in a single process, -and- gets a concurrency model that doesn't suck, baked in distribution that doesn't suck, better VM introspection, fault tolerance second to none, tail call optimization, soft real time performance, higher order functions, and glorious, glorious pattern matching.
Or maybe not, but just sayin', you're making assumptions in trying to defend Java there. It's enough to say "You don't have to write Java that way", and then find reasons why Java stands on its own ("It's the lingua franca of software development", and related, is the best I can come up with, but eh), rather than attacking a strawman by assuming what traits the language(s) he switched to are and comparing Java against those.
There ya go then. Though, Clojure doesn't have most of those additional things I listed either, as most of them are requirements of the VM rather than the language (it does, however, have a slew of other things that make it awesome, and it has the language level features I mentioned)
I thought the same until I actually tried working on big projects with other, simpler languages. Threading can be replaced with process forking in those GIL languages, and everyting else is just so much better. 4 years after ditching java I will never ever want to write code in it again.
The world has a few other languages besides Java, Python and Ruby. Not every language has a GIL, and its the damn 2010s and we've got other concurrency models worth trying beside the 90's staple of mutex-littered multithreaded apps or callback hell.
My latest Java project is cca 70,000 lines of code. 1 XML file (because the first developer didn't know about spring annotations). No need for eclipse plugins, in IntelliJ I install only two: vim-plugin and lombook.
I'm not saying the Java world isn't too complicated (it is) but if you know all the possible problems -- you can keep it sane.
Nowadays, I'm moving slowly to golang. But I don't think I will stop programming in Java. I still think it is a nice language, but if you inherit a project from a "enterprise Java developer" who thinks in EJBs, then just run away :).
Overall, I've found a lot of the AWS documentation to be missing essential information or examples, or vital bits scattered across the AWS web properties. For example, setting up multi-instance Docker containers was not trivial; essential IAM policies needed to get AWS to work were scattered in marketing information. I think AWS has a great collection of products, but it is definitely not turnkey.
The article headline captures the essence of the issue exactly right "the AWS Java DOCUMENTATION is ...".
You can write Horrid examples in any Elegant Language and conversely you can give elegant examples in Brainfuck. The person(s) writing the Java example fell into the classic Java trap that mid-level Java folks fall into. Namely that trying to follow SOLID while writing an "Hello World" Distributed Java example.
They lost sight of the fact that the end product is supposed to be an EXAMPLE. This is why the senior devs (Java or otherwise) make the big bucks. Not because they can write complex hairy code but precisely because they understand when NOT to do it (almost always - YAGNI).
I followed the same guide when I learned SWF and used it in a project. I appreciated the detailed guide and was amazed by the flow framework. Because of flow's magic and the steep learning curve, I believe SWF might be the most underestimated AWS service. I agree that the java flow framework is complicated but very powerful as well. Seems the ruby flow framework is simpler to use (http://docs.aws.amazon.com/amazonswf/latest/awsrbflowguide/h...).
AWS documentation is by far the worst I've ever seen. For all their services. It's unsearchable (through Google or otherwise), most of it no longer pertains and has been superseded but they won't tell you. It's split up into many different places that are not linked together, so when you think you've found your answer, you probably haven't. And of course, it never covers the truly hard problems like all the bugs in their platform. Total and absolute shit.
in Java you always have this.. Impl Impl Impl.. Impl everywhere. I feel it's being passed from a senior and a junior dev who interprets the "program to an interface" idiom. Its so annoying to follow a code that is deeply nested with calls on a very basic controller call. It seems to be program to think that calls will be dynamically "replaced" and "reloaded".
So much complexity in the Java world and I don't know if its because of the language has not evolved throughout the years or the quality of developers (majority) that it has produced. Reminds me of this https://github.com/EnterpriseQualityCoding/FizzBuzzEnterpris...
We also have the same developers coding in javascript. I literally need to refactor once a 7k+ lines single ExtJS controller. Yeah Extjs is so popular in the enterprise java world.
To be fair about Interfaces and implementations- when you're using DI and Aspects, or really any type of framework that is going to be proxying your classes it is a good idea to code to an Interface, even if you only have one implementation.
I read a few great articles on this a long time ago and unfortunately I don't have the references at hand, but it was compelling enough that I Interface anything that's going to be proxied or injected.
That being said, I REFUSE to call anything an Impl. If I have an interface called FileSavingService, I'll have a LocalFileSavingService or an S3FileSavingService, but I'll chop my fingers off before I add Impl to the end of them.
I generally like coding things to interfaces, as you often implement the interface twice; once for the real system, once for unit tests of other dependent components. Not using interfaces leads to either compiling test code into your production binary, or other more-horrifying hacks (mocks!).
I mean, to be fair the quality of the C# apis for AWS is also a lot better than the Java apis. There are a few okay ones like the SQS library but...
It is not just the documentation.
Having recently been embedded into a very large corporation with a lot of legacy tech platforms, I have to say I've heard people praise the AWS Java documentation, apis and design patterns.
I'm not sure if this sort of thing is the done thing, but I flagged this because I think the title may flag the front page on some work proxies/filters/monitoring systems. The title just needs minor redacting/editing, that's all.
I've never used Java (not counting experimenting with Bluetooth in J2ME for lulz), but I've head a lot of opinions and thoughts about it. I wonder if (especially considering the parent thread off the above link) Java's tendency to encourage AbstractSingletonProxyFactoryBeans (see: https://news.ycombinator.com/item?id=4549544) is simply a way to fill developers' heads with so much indirection* that we simply cannot grasp that it ultimately makes no difference.
* - indirection: my poor word to describe having to constantly resolve what something means, and cross-reference information, etc - stuff that swamps and/or overflows our mental stacks really quickly.
Of course, the above being said, the elements of the language seem elegant - patterns and models and the like providing us with "everything has a place and everything in its place," if you will - and of course the encouragement in exponential verbosity looks really good in our progress reports.
Until you ever start interfacing with enterprise libraries.
As a novice, you'll just decide not to interface with them. Then you decide to build makros that make interfacing with them simpler, and then you build just another abstraction over them, using the very same AbstractSingleFactoryBeans that the library itself used.
The reason why Java managed to be usable by modders of Minecraft — kids who've never seen code before — and enterprise people at the same time is that Java gives you the tools to build all the abstractions in simple ways, inside the language.
In other languages you build a small hack using the preprocessor, or even decide to build a DSL. With Java, you just build an abstraction over the existing things.
As a result, you have 20 years of abstractions right on top of each other.
Oh, okay then. TIL. I have to admit that I've not really looked into Java that much, probably due to being tangled up in the opinions and rhetoric. Thanks for this perspective.
I guess I need to explore the language further! (And ideally with moderation and discipline, it would seem.)
Java is missing some features I'd really like, but there's nothing actually wrong with the language. You can write a C-like interface in Java and it will come out slightly simpler because of the garbage collector. It has features beyond that, of course, but I think you only need functions, structs, and namespaces to make clean code, and Java has at least all of those.
I think the problem can be explained in two parts, having little to do with the language itself. Schools teach inexperienced programmers design patterns without teaching the absolute basics of code organization (refactor reused code into functions, move functions that deal with a class into that class) so they enter the workforce and try to solve easy problems with design patterns. Intermediate programmers in boring Java shops are judged by the complexity of their code, so the ones who never get past the solve-everything-with-design-patterns stage become senior programmers who mentor the newbies.
Half of the world's bad, professional programmers use Java (the other half use Javascript), so they're quite effective at drowning out the good Java programmers. The whole system becomes clueless newbies cargo-culting together things that did well in performance reviews.
I made it about half way through when I realized I'd rather put a bullet in my head. The JVM is pretty neat and Java has a huge ecosystem but it's misery like this that I can't tolerate. Using it from within Clojure can make it palatable.
The person who wrote it probably gets measured on the amount of "work" he/she produces.
The more detailed the tutorial, the more "work" he/she has accomplished, at least according to the rules/processes internal to the department in Amazon.
The person writing the tutorial has to make it extra complicated. Otherwise she/he will lose her/his job because other documentation writers will do a "better job" by writing more "detailed" tutorials.
Another scary thought: What if the person thinks this is clever and admirable code?
I think many of us spend a lot of time in our ecosystems and we might lose sight of what's outside. If someone spent all their days writing complicated Aspect-Oriented code then maybe to them this is just second nature and the right way to do it. What if you wanted to add more features, after all?
I wouldn't go so far as to imagine perverse incentives from management. It could just be a cultural bubble that the person works in or they could work in a place where most problems are far more complex and these sorts of solutions make sense.
It can be difficult to remember who your audience is, what they know and what they need to be told.
The C# example being discussed, as well as the Java example at the end, are examples of using that SWF API directly. The SWF API is indeed simpler for trivial examples. Flow is a power tool that handles complex workflows better than any alternative I've seen, but the framework itself is complex and incurs cost to set up and use. The documentation could do a better job of explaining this, and of providing Java API examples.
The Flow framework provides something that's a mix of Java code and a domain-specific language for building SWF applications that's expressed as Java code. (For an analogy, consider EasyMock.) It's hard to explain Flow concisely, but if I had to try I'd say, "You write code that looks like it's procedural and runs on one machine, and Flow converts that into a distributed workflow running across of fleet of machines, all of which may be stateless". Flow threads the state into and out of SWF for you, making it possible to express distributed workflows at a higher level of abstraction. Some examples are in the AWS Flow Framework Recipes: https://aws.amazon.com/code/2535278400103493
To achieve this, Flow uses fancy code weaving & AOP techniques. This makes it more complicated to set up, develop, and test. Flow pays off however once your workflow is more complex than a simple linear workflow. You could, for example, process data with a distributed map/reduce pattern in a few lines of code in Flow. (Source: built production systems on SWF with and without Flow)