[Closed] Software Development - where next?

Machine learning/data science? Lots of algorithm stuff in that, it's not actually magic and needs clever people to make it work.

Head First

Is likely to be an interesting read for you. Are you looking to relocate, or stay locally? FinTech might be worth a look as some will be developing new tools as well.

Those are not that common these days, because most of the hard stuff has already been done and is available in a library or an OS project. Unfortunately, what you're asking is like someone who's just got a degree in automotive engineering and is looking for a job with an F1 race team but the only jobs they can find are Kwik Fit. Just keep looking, you'll find something interesting eventually. I went through loads of jobs early in my career and tried many things. Only took about 15 years to find one I like, and it's not because of the code I have to write 🙂

One option for you would be to plod on as you are now then look at contracting in a few years' time. Same shite, arguably worse, but if you're frugal you can work your contract then take lots of time off and travel the world etc etc. Depending on your commitments ofc!

Those are not that common these days, because most of the hard stuff has already been done and is available in a library or an OS project.

I understand this completely. I had previous career in acoustics and noise and picked up a few second hand books written in the 70s and 80s which are full of fascinating engineering for noise control. However these issues have now been fully solved and integrated to working practice so specialist knowledge is no longer required.

@toby1 I've heard of that book, will get hold of a copy

I do a lot of work in embedded systems. (i.e. writing stuff for microprocessors etc)

In that environment you are often writing fundamental stuff like searches because you typically don't have an operating system and you don't have the space to pull in pre-written libraries even if they would get past safety testing (which most won't).

BUT.. it's a big step to that from Java. Most stuff is still written in C or a specialised subset of C++.

If you are interested though then picking up an Arduino and playing with that is a good start.
Once you get the hang of it try stepping outside the Arduino IDE and programming it directly.

Platform engineering is something that may be of interest, i.e. in order to have a good CI/CD pipeline, more and more automation and infrastructure is built with code. It may not be the most glamorous of work, but it can be well paid as it's not a widely available skillset and there are always new things to learn as the tech under cloud platforms is constantly evolving.

@GrahamS

I have considered embedded systems, in fact I'd like to get hold of a system that I can programme for audio DSP use, e.g. filters or basic effects like reverb. I have done embedded programming before with C. Although, this was a while ago at uni. I've also used Matlab quite a lot for data processing and modelling of acoustic systems.

I'm only really working with Java as it was easy to teach myself and jobs were available. A lot of my day to day work isn't even in that though. I'm more working out ways to get out of the corners that the unnecessarily complex solution design has painted us into. The wider scheme aim is a deprecation, however it's quickly descended into a large cut and paste exercise. The projects are focused solely on selling new products, with no consideration to improving the code base or making it extensible and flexible. There is no holistic or systemic thinking. The upside is there will be work for years and years to come if I stay where I am now.... but it will be more of the same

My huge problem with the typical interview is that it includes algorithmic stuff like that, because unless you're in a specialist field (deep embedded, or actually writing new algorithms) the solution is to use the standard library of whatever language you're in to do the operation. Encouraging graduates to write their own sort algorithm is not a good thing.

Far more useful is the understanding of what the tradeoffs are with various algorithms and data structures.

The skill in the modern era (at least in my area which could loosely be described as 'enterprise') comes from integrating all the different systems, platforms and techniques so that they don't all fall on their arse when you release the software, by understanding how things really work and predicting what'll happen if you do it wrong. And its a skill that seems to be in short supply. People who can write Java code are ten a penny if you recruit overseas, however that on its own isn't enough.

I ended up in this area fairly randomly after being a Java developer - but then, I think this kind of work is where most Java gets written (purely anecdotally, I have no idea really). My best advice is simply to change jobs if you can, and if it's suitable, and you'll pick up enough experience to get yourself into a position you like.

On the other hand, I'd say don't change jobs now, because the economy could fall on its arse in a Brexit-related way, and having more than 2 years service at a company gives you a better payout if you do get canned. This is the main reason I'm not applying for a very tempting looking job near my home - I have over 7 years at my current place.

Encouraging graduates to write their own sort algorithm is not a good thing.

Yes but it shows how they solve a problem and how they work - it's not the actual algorithm that's important.

I’d appreciate some pointers on what to do next (if anyone has any)….

If you want to be writing ultra efficient code then certain parts of fintech will be a good match but think they will be a lot lower level than Java and need the maths skills etc to support it.
As others have said 99% of the time wont be needing to write ultra efficient algorithms for two reasons. First its often easier to just get a library where someone has done the heavy lifting and secondly the cost benefits often arent sufficient. Occasionally I do end up doing really detailed stuff when there is a obvious bottleneck but its rare.
One thing might be worth doing is signing up on hackerrank and start working through their exercises

Yes but it shows how they solve a problem and how they work –

Or how much time they have spent reading up on the common interview questions and solutions.

One thing might be worth doing is signing up on hackerrank and start working through their exercises

Use a throw-away address if you do because I got an avalanche of spam after I signed up to them.

I had forgotten about HackerRank, I think I'm already signed up... but not used it in a while.

As others have said 99% of the time wont be needing to write ultra efficient algorithms for two reasons. First its often easier to just get a library where someone has done the heavy lifting and secondly the cost benefits often arent sufficient.

I think it's good to have the insight as to how these algorithms fundamentally work or might be structured, especially if a library doesn't quite do what you expect.

An Open Source project can be a useful way of getting both interesting challenges and also getting peer review. Contribute a task scheduler to the one of the open source unices and you'll get a good grounding in all sorts of processing algorithms, both theoretical and real world.

+1 on doing (or even just reading) some open source contributions.

If I was inclined to learn any Java, that's where I'd be going.

Read Clean Code by Robert Martin if you have not already done so.

I think it’s good to have the insight as to how these algorithms fundamentally work or might be structured

Yes and also you can draw analogies for the problems you do have to solve at higher levels.

senior software engineer. and been in the same job approx 25 years. mainly c/c++ from code in custom hardware to mainly application software only. (also used lots of other languages and web technologies as the market/job has evolved).

we spend a lot of time doing QA as our projects hit deadlines then a spurt of dev, and it loops. as someone said above you want the fun parts without the bits that arenyt so fun. this is isnt academia where you get to play all day.

we have a lot of difficulty finding competent programmers. we literally cannot fill jobs.

as someone said above you want the fun parts without the bits that arenyt so fun. this is isnt academia where you get to play all day.

I don't think I see where someone has said that.

I have no objection to QA and testing at all. I prefer to follow the test-driven development method too (unlike some colleagues). However, all my team does now is QA, there is no dev. This is because the system is so poorly designed that any new "product" passing through our service has to be 100% re-tested from the ground up, even if it was version 2 of an existing product. The testing overhead is getting out of control. This is because the solution designers don't really understand (or want to understand) how the code base works. They prefer to introduce entire new concepts rather than evolve existing paths. This means existing tests of the service for version 1 of a product have to be completely rewritten for version 2.... and there are multiple services so every team is the same.
The net result of this is that my coding skills are not getting used or developed.

yup i know that story, fortunately here competent modularisaton has eased most of that pain.

i think everywhere has its own approach and culture. integrating with teams running various or no AGILE or whatever methodology shows that up.

how about starting your own company? 'the lean startup' book changed my view on a lot of things.

rossburton

My huge problem with the typical interview is that it includes algorithmic stuff like that, because unless you’re in a specialist field (deep embedded, or actually writing new algorithms) the solution is to use the standard library of whatever language you’re in to do the operation. Encouraging graduates to write their own sort algorithm is not a good thing.

Far more useful is the understanding of what the tradeoffs are with various algorithms and data structures.

I see what you're saying but at least it shows that people can actually programme, rather than just doing a bit of copy pasta from stack overflow.

We were looking for SQL devs recently and the amount of people who claimed to be SQL developers but couldn't understand how to optimise a simple query blew my mind quite honestly.

Thanks for all your input so far.

Here's what I'm wondering today. We are currently repackaging/restructuring the service my team works on. This is no small task. Legacy code that's seldom changed is either being put into libraries or the need for it removed altogether. It's also being converted to use Spring boot and removing it's reliance on Websphere. All of this work is being done "off the books". There is no business project for this, so work is being done as and when it can be. However it will save the business quite a lot of money, e.g. removing the idle time spent waiting for a deploy to Websphere would free up huge amounts of developer time.

My question, is it normal for work like this to have no support from the business?

My question, is it normal for work like this to have no support from the business?

Certainly isnt unknown. What is rarer is having the time to do it.
My team has a long list of improvements we want to make but generally only are able to when something blows up and we can point at the list and say that would have helped.

My question, is it normal for work like this to have no support from the business?

Very. If it goes well then you will get little credit. If it goes badly then the blame will be on you for doing stuff that wasn't planned for and introducing unnecessary risk to the project.

On the flip side:

Engineers generally hate seeing untidy solutions and enjoy reworking stuff in this manner. But from a company perspective it isn't always a good thing.

Always ask: can we sell it? Does it make the product better? Does it save us money?

Recognise that engineers who are "business focused" go a lot further than the ones obsessively fettling with technical stuff for the sake of good engineering.

(Personally I enjoy the fettling)

Perhaps this is different where the software is the "product", rather than being an enabler to sell the "product"? The software is very much the enabler here.

I can see there would be broad spectrum from tinkering at the edges to full system reconfiguration. This current work is towards the latter end. If those in charge thought more about how the system works overall and decoupled the work from the projects that sell/manage the next new product type, they could save the business a lot of money, reduce the dev head count and deliver new functionality quicker. Because this isn't going on, I think there are quite a few people resisting/ignoring the benefits of any changes in order to conserve their job.

Have a look at Python (used for many things such as machine learning and data analysis and forecasting

Find a domain you want to work in - the FinTech domain and it’s reliance on risk analysis and authentication is a growing area

Perhaps this is different where the software is the “product”, rather than being an enabler to sell the “product”? The software is very much the enabler here.

The pitch will be slightly different but generally there is the same need in both cases.
In the former you need to convince management why it is either a selling point to the end customer or has internal advantage eg will make it easier to add new features in future and so sell more.
In the enabler mode for the former the management will also be the customer so if anything a bit easier.
So when proposing changes you need to able to word it to them in terms of how they will gain. Not always easy and they will often ignore it anyway. I have just finished up a "told you so" for an item which whilst not an immediate problem had the risk of becoming a serious performance issue. Couple of years down the line it definitely ticked that box.

In the enabler mode for the former the management will also be the customer so if anything a bit easier.

I think the problem here is no one bothers to pitch to management anymore and hence the "non-functional" changes are made in the way I describe. It could be easily fixed by decoupling the workstreams, with a fire break between business projects and system projects. Currently the business projects carry right on through.

I've just realised that in the 3 years I've been here, there's never ever been a discussion about the system architecture, no presentation about future aims, no feedback gathering, etc. That's actually quite strange for the type of company it is.

I’ve just realised that in the 3 years I’ve been here, there’s never ever been a discussion about the system architecture, no presentation about future aims, no feedback gathering, etc. That’s actually quite strange for the type of company it is.

Sounds like quite a disconnect between the various groups. I'd advocate making what you are doing more visible, but also asking people about the strategy and architecture. You sound like a sensible person, sometime it just takes someone asking some sensible questions to bring about some change. There are a number of things where people do things the way they do them, because 'That's the way we do things here', but someone asking if that things really needs doing can cause a conversation about it and may lead to a change of practice.

I know it sounds a lot like I'm telling you to do stuff you may not be comfortable with, so feel free to disregard it if you don't want to be that guy 🙂

My question, is it normal for work like this to have no support from the business?

Pretty unusual in my experience. Who's paying for you to do the work? Anywhere I've worked, nothing happens without a budget, and you don't get a budget for work unless the business have signed up and someone is owning the job.

Going back to your original question, in Edinburgh, financial services dwarfs everything else in the IT job market, but it's unlikely to offer what you are looking for. FS is about selling financial products, not writing innovative software, so as others have pointed out, the work is more about plumbing together third party systems and services and minimising what you have to write from scratch. There is inevitably lots of QA and testing because the impact of getting something wrong is significant, if you have millions of customers.

Left field suggestion, if you want to broaden your options and look at different forms of creative problem solving, is to investigate opportunities in analysis, architecture and design.

Who’s paying for you to do the work? Anywhere I’ve worked, nothing happens without a budget, and you don’t get a budget for work unless the business have signed up and someone is owning the job.

In effect, the business focused projects are paying. However, all the business requirements are still met and obviously take priority. Time on projects is tracked by a weekly timesheet and is normally only as granular as 1/2 days but is mostly logged as full days. However, there are (bonkers) moves to start to log time to tickets in JIRA in addition. This is a doubling of people's reporting requirements and they're going to get a shock if they try and reconcile the timesheet and JIRA data. There has been a push back against the JIRA time logging. They want the data to show current resource availability and better predict timescales for future projects. However the time logging in JIRA is not taking any account of the complexity of the tasks vs the time they take. As the complexity isn't noted (i.e. no story points are added to the tasks), the "time taken" data they are gathering will be pretty poor for predicting future project duration.

Time tracking in Jira suggests someone is looking at the cost of your department, can be a goods thing, focus on the things that add value to the business, can be a case of them looking at their implied costs vs benefits. Maybe that you will need to find a new job before long anyway 😉

If it's anything like the software maintenance contracts we have with suppliers, the JIRA time logging will be related to effort which will be billed to the customer, and is sod all to do with company time sheet booking codes.
But then we're a customer, so we're mostly tracking that they're not making ridiculous estimates for effort to close issues.
What they put on their timesheet doesn't concern us. What they put on the invoice does.

Time tracking in Jira suggests someone is looking at the cost of your department

Indeed, a colleague has just been asked to justify time spent on a project back in September. It'll all come out in the wash eventually that their timesheet data is bobbins.

I've just spent the same amount of time this morning tracking down a project code to put time against as I spent on the project itself yesterday

The The Algorithm Design Manual by Steven S Skiena is (IMHO) a good place to start for an overview of the world of Algorithms. I especially like the little stories he includes that illustrate real world situations where selecting the correct Algorithm is key to solving the problem.

Coding interviews that test for knowledge of a particular Algorithm have never seemed sensible to me, it's a bit like rolling a dice - if candidate A happens to have encountered that particular trick they look good while candidate B who hasn't is stuffed.

I’ve just spent the same amount of time this morning tracking down a project code to put time against as I spent on the project itself yesterday

Whilst its calmed down a bit thanks to a multi national team and a mix of different systems for a while had to use three different time recording systems all of which had noncompatible codes.
Hours of fun.

Ours was a pretty end of chain R&D outfit, so there wasn't the same pressure to justify a project details based on business case -- it was a big funding up front, and then paid for over the course of a couple of years. At least it was like that at the end. Was mostly C/C++ then Java. Key thing is to keep learning, new techniques or approaches and methodologies.

Edinburgh / Central Belt development is a pretty close knit group, there's usually someone that knows someone else. I've met (briefly) @GrahamS whilst going for a job interview. IIRC there were little algorithm teaser questions in that interview - I'm not sure I got very far..

Nowadays I'm more QA for a small lifestyle software dev business, what I learnt in the latter days in Edinburgh has helped here to be able to see things from client as well as dev perspective -- and to get repeatability in quality control. Something that wasn't really on the horizon back then.

.. sorry for the ramble !