How healthy is your Ubuntu system?
Now that edgy is released, many people again are complaining about having a hard time upgrading. And as with every release most of the issues found are caused by unhealthy systems. What makes an Ubuntu system unhealthy, you’d say - well here are a few examples:
- Automatix/easyubuntu/fasterdapper
- Third party packages
- checkinstall’ed or alien’ed packages
If anyone has other suggestions for this list, please comment. For feisty I want to turn this list into an ‘Ubuntu health checker’ that can tell you where you may find problems and possibly how to solve them.
Just wondering, how exactly do Automatix or easyUbuntu make your system “unhealthy”?
Because it does all kinds of crap to your system. We’ve spent countless hours helping people undo damage that was caused by automatix.
[...] Over at Steady as a rock, Dennis asks the question: “How healthy is your Ubuntu system?” [...]
I totally agree on the part saying that automatix/easyubuntu is unhealthy (from an intellectual point of view and from a system point of view … if you want a system already configured stay with Windows, they are good at it). So my question will be : “Why automatix/easyubuntu and alike are not banned from ubuntu forums ? We need to say to newbies that these scripts are unhealthy.”
By the way I upgraded only two boxes and that was smooth and painless ;)
you are right, but without a tool like easyubuntu, ubuntu is just not ready for the masses.
I heard Ubuntu want to integrate somthing like this. I hope this will happen, because it is needed !!
Normal PC users don’t want to look at the internet every hour an check what they have to do to play mp3, divx, …, …
tomm: I absolutely agree, and so do many other Ubuntu developers. But a patchy ’solution’ like such scripts may be easier to implement, but it’s much, much better to properly integrate it into Ubuntu. See also https://features.launchpad.net/distros/ubuntu/+spec/easy-codec-installation
You’re right that automatix et all is the wrong solution to the right problems. But why does nobody engage the authors of such, especially when you see things like the following from the FAQ:
” Is Automatix safe ? Folks in #ubuntu on IRC keep telling me it isn’t
* Yes it is perfectly safe. Thousands of users worldwide use Automatix every day without any issues. ”
Instead of correcting and inviting people to participate in the community process, we see developers either a) smug posts suggesting “I told you so” despite having not actually loudly protested in the past, or b) committing the same 3rd party package sins.
For example, I saw a post on the Planet a few months ago where a developer packaged up the breezy artwork for dapper. I gave it a spin and left it to fester until yesterday, when I upgraded to edgy and suddenly ubuntu-artwork was trying to overwrite a file owned by ubuntu-artwork-breezy. I thought that surely packages directly from someone syndicated on Planet would be safe. I was wrong.
And that’s okay with me. Because even though it totally tanked the dist-upgrade, I fixed it on my own. I might have fixed it faster if I had submitted the error update-manager gave me about being unable to calculate an upgrade path, but nothing outstanding to me showed up in the logs so I simply prodded on, and eventually fixed it. It might be nice to see a list of packages installed that have no installable candidate from sources.list, I suppose.
It might be time for UbuntuSchool to focus on a initial packaging class, complete with homework of sufficient complexity to expose typical problems seen in the field, complete with screencast.
jldugger: I removed the huge link from your post, hope you don’t mind ;)
As far as stability goes: only things in the archive of stable Ubuntu releases are to be trusted. Nothing else.
[Edit by Dennis] Lamia, this is not the place to rant about shortcomings of Ubuntu. We all know that linux is not yet the most excellent multimedia platform and we work hard to solve that problem.
I agree that automatix and the like are the wrong solution to the right problem as someone said.
The correct solution being integration into ubuntu fully.
I used to use automatix though, but i no longer do. there is nothing automatix has to offer that i cant get from the repos.
Now, in edgy, it’s easier than ever. applications > add/remove. then type “codecs” and check the first two boxes.
I was also real impressed that flash installs correctly from the install built into firefox, where in the past that didnt work.
I do like the thought of a Health Checker, the problem is that nobody will use it. Why, because it takes effort…
These things should work preventive, for example making apt refuse repositories from other distros/versions, so Edgy would refuse a Debian or Ubuntu Dapper repo from being added to it’s sources.list.
And for example having the upgrade tool forcefully remove all third party repos from the sources.list before beginning the upgrade process. (or something like that).
Dennis,
I agree. In fact, on my Ubuntu Wiki FAQ page I wrote up a while ago a way to correct these problems to ensure as close to a flawless upgrade as possible.
Direct link: https://wiki.ubuntu.com/JoeyStanford/MyUbuntu/UbuntuFAQs#head-0cb087bbafb069a2239810ed579d081cd1521c37
Pascal: the dist-upgrader already removes 3rd party repos. That’s exactly why I cannot use it, since that also disable the usage of apt-proxy.
That’s right, blame the user.
I’m sorry, but I just got through the very worst upgrade experience of my life, and that’s including my Windows and OS/2 experience. Yes, I had installed a couple of packages via automatix, but I painstakingly tracked down the cause of every issue I had, and NOT ONE of them had anything remotely to do with Automatix. In fact, they were mostly issues with core packages.
For instance: the dist-upgrade itself failed repeatedly, with various errors. I had to manually restart ‘apt-get dist-upgrade -f’ over and over again (this was after disabling everything but the core sources). You can’t tell me that the x11-common package reporting a “invalid nice level” and refusing to install has anything to do with Automatix.
Or what about the fact that the upgrade broke my swap partition? Turns out it failed to include my swap partition configuration in the new initramfs. Not having swap was bad enough, but it also broke hibernation on my laptop. A number of people have reported this same issue.
And then there was discovering that the half the auto-generated font paths in xorg.conf were wrong.
I originally moved to Debian years ago because I was unsatisfied with the upgrade experience in other distros. I didn’t like having to effectively reinstall with every release. But with Edgy, it’s sounding like the people who did a clean install had a much easier time of it than those who tried to upgrade. Sure, some of the outlying issues might be caused by “dirty” systems. But when core packages refuse to install or break pre-existing functionality, there’s something more going on.
I admit the above was pretty much pure bitching, but here’s the thing; I’m a developer who could potentially, and would like, to help fix some of these issues in the future. But this “blame the user” attitude turns me off, and makes me less eager to help. The fact is that there are real issues with the Dapper->Ubuntu upgrade process, where the relevant scripts simply fail to fully bring a system’s existing configuration in line with the changes that have come with Edgy. I have the skills and the will to help, but when I see “it’s because your system is unhealthy” it makes me think “why bother?”.
Will using the Seveas repository make my Ubuntu system ‘unhealthy’ too?
http://blogs.ubuntu-nl.org/dennis/2006/10/19/as-promised-edgy-seveas-is-finally-here/
Avdi: enabling third party repos will give you such errors. I’ve only seen a few 3rd party repositoris that function flawlessly.
Sander: Yes, potentially. Especially if you use the drivers component. That’s why there’s a warning for them. I’m *extremely* careful about not breaking upgrades though, which cannot be said for most others.
I personally had XGL/compiz running, so I’m happy to accept it was my own fault and had to spend over 3 hours doing the upgrade with force.
But blaming all the upgrade problems to 3th party’s doesn’t seem right.
I know users that I tried to convert to ubuntu who had fresh dapper installs (not even a week old without unofficial repositories or pretty much anything extra installed) upgraded to edgy systems that showed nothing more then a black screen.
So basically we should not install anything other than what is provided in Ubuntu repositories????? If you believe this will be the case you live in a fantasy world.
Well i am sorry but Ubuntu will never be a success if it falls over at the slightest variation to what is expected. If Ubuntu cant stand up to modification then it is a joke. Considering all the shit MS gets for its systems, maybe we are just starting to realise the difficulties of letting a large number of non-technical users, having their desktop clients out on the net in what is a wild-west, where they will install anything if a significant minority recommend it.
And repositories are not always up to date with the third-party releases, so in these situations it is resonable people will install their own packages.
Ubuntu should be able to protect parts of it system that are vital for smooth updating and operation. The users are not at fault in this case, they should have the right to install what they like, and Ubuntu should have the ability to protect itself from harm that these packages could present.
If Ubuntu is about people and freedoms, then wouldnt it be ironic if Ubuntu was a system that was completely locked down, and content had to be spoon fed to users from the Ubuntu overlords? lol melodramatic but sometimes the whole repository gig feels that way.
Maybe it helps if installing things in /home instead of in /usr would be promoted more and made easier. Things like the Flash plugin, Java and new versions of applications can be installed in /home to offer the same functionality, but not damage the rest of the system. Is that possible for gstreamer codecs too?
Matthew, no I’m not saying that. I’m saying that users should be warned when installing 3rd party packages, not that they should be forbidden to do so.
It is a difficult balance I think. I went through 3 upgrades to Edgy and only one (which was knot3) had trouble.
I had the “black screen” and X did not work on the one early test. For many the solution to this seems to be “install from scratch” which was a last resort to me. I boot to console though, not gdm, which might have been the saving factor. Really it was easy enough to fix, just install the xorg packages and I was back in business. There was also a problem which prevented shorewall from working that was not fixed till RC1, but that is what betas are for. All in all I would consider my upgrades to be painless.
OK one other problem, I use an atheros card as an AP and had to gut my network config a bit and install the madwifi-ng tools in order for that to work again. But that is reasonable for an upgrade, and the excellent Ubuntu forums quickly pointed me to what I needed to do.
Upgrades are critical due to third party applications though. If all I used were stock packages I would just backup and restore my home directory. As it is a backup of opt and home would get most of it, but not everything. VMWare, configs for X, ssh, shorewall, tinc, etc… Over time it becomes impractical to “start from scratch” without increasing the effort of the upgrade manyfold.
So perhaps there could be some middle ground? I do not think it reasonable to lay the responsibility completely on the development team. It is just not possible to take into account some of the creative things we users do to our systems. The thought that came to my head is a way to update an existing from the live CD. This way if a conventional upgrade makes the system unusuable perhaps the CD can finish and or recover the installation. At the very least it would be nice to do a fresh install from the CD while preserving users and home directories.
While I think it likely most of the install problems can be tracked back to the users, it would perhaps be helpful to make the sytem more capable of handling what the users break.
We are all avoiding the real issue here.
Everybody is doing the COMMON CUSTOMIZATIONS.
So let’s just integrate them into Ubuntu.
I don’t care about the legal status.
They need us to march down the streets to get the right to do so? I WILL.
You need me to rebel to my goverment? I WILL.
You need me to kill any lawyer making an issue of my free desktop being interoperateable with the rest of the world? I WILL.
We are all technically breaking the law. Lets do it out in the open.
I AM NOT ASHAMED OF THIS AND NEITHER SHOULD YOU BE.
So let’s do it out in the open: we have got nothing to hide. If we are breaking the law; the law needs updating.
For me the last couple of Ubuntu releases have been the same story, something either breaks or doesn’t work quite right or both. I feel that the upgrade process is something that Ubuntu really need to work on as upgrading between Ubuntu releases certainly isn’t in Ubuntu’s strong suit currently.
Probably the worst part of this is the number of users who simply give up and decide to give some other distro a go while they have the chance (they have to do a fresh install of Ubuntu anyway so what the heck). How many of these people then stick with their new distro and never return? What about the loss of competent users who have been using Ubuntu for about 6 months who are just getting to the stage where they could be really useful to the rest of the Ubuntu community (writing docs, providing support on irc/forums/mailing lists)? Surely this is impacting badly big time on Ubuntu due to the number of cases that keep poping up.
The upgrade process is something Ubuntu really need to address IMO, it’s not just as simple as passing the buck onto third parties or other causes (a classic example of this was the recent Nivida binary blob security incident). Its in your distro that you provide and you have a duty to those relying on you to get it right, otherwise people will begin to drop you like grandmas underpants even if the GPL states there is no warranty with the software.
If these issues are not fixed then I doubt Ubuntu will ever be able to close bug #1, imagine the outcry if Windows upgrades went so badly?
We get this all the time - and mainly from the forums, but there is some stuff on the wiki too. It would be good if we had something official-ish we could point people to when they ask why they shouldn’t install automatix etc.
I’ve just seen this (http://community.livejournal.com/ubuntu_users/141068.html) on live journal which also looks problematic. http://reconstructor.aperantis.com/ - called reconstructor. Naturally people want it to be connected to easyubuntu etc so they can change everything without knowing anything about it.
[Edit: no profanity please]
I think that Ubuntu has teo big problems:
I have seen people using sources.list files which mix repositories from all of these:
- Ubuntu Dapper
- Debian Sarge
- Debian Etch
- Debian unstable (Sid)
- Debian experimental
- backports.org (which contains sarge backports)
- dotdeb.org (which contains sarge backports)
- debian-multimedia.org (which contains packages for the debian)
- various other unofficial reposiotories which contain packages for the Sarge, Etch and Sid.
Do you seriously think that mess like that will ever work properly? It really is not any surprise that upgrading does not work when you use sources.list like that and install packages from all unofficial sources you can find. I’m surprised that your system even works if you have sources.list like that.
Second problem is that there is just about 2000 officially supported packages in the ubuntu and vast majority of packages in the Universe repository are just rebuild packages from the Debian unstable. Those packages are mostly not maintained by ubuntu developers at all, those are just rebuilds. Debian unstable is development version, so it’s packages still have bugs.
Eg. Debian Etch repositories have about 18000 officially supported and tested packages. So if you use Debian Etch, you don’t need to install so much stuff from unofficial unsupported sources. So it’s not any big surprise that Debian is so much more stable than ubuntu is.
Testing is wery important part of the software development, I think that ubuntu developers are badly neglecting it. And this is the result.
Ofcourse Etch is not released yet. Current stable is Sarge.
I’m sorry if I hurt someones feelings, but if you want stable Debian-style system, then use Debian. Just wait a bit until Etch is released. It does not take long anymore, maybe 2 months or something like that.
One problem is libraries which are used when building binaries. It’s easy with Windows because there is just one Windows XP etc. But eg. ubuntu and Debian do not neserrarily use same version of library to build the same binary. And that can cause severe problems if you mix softare which is build for different distros.
It’s true that you can’t blame just users. Insufficient testing is the biggest reason. Ubuntu developers should seriously rethink it how often Ubuntu will be released.
Mike: Any ideas on howto make it more robust are very welcome, really!
Meneer R: Sure, let Ubuntu/canonical be sued, that’ll help! And please don’t use so much profanity, I’ve edited it away.
miksuh: I’ve seen worse than that list ;)
How about autopackages?
One of the MAIN reasons I no longer frequent the IRC channels for Ubuntu is this hellfire on third party applications as being evil. You say you’ve had all this pain with users….well WELCOME to the world of tech support, where the user will NEVER EVER EVER completely come clean on the problems that they have usually dumped on their own heads.
You want lilly white upgrades, then you need to start looking around for “deh plane! deh plane!” because you are most DEFINITELY on fantasy island.
I’ve done several upgrades with automatix’d and NON automatix’d ubuntu systems, and the problems on one usually happened to the other in most cases and NEVER have I seen problems directly related to what you are talking about. I’v also had a couple clean upgrades WITH automatix’d systems.
Its an EASY thing blaming it all on the 3rd party, but when that 3rd party is doing the community a SERVICE by getting these machines up to snuff in an efficient and simple manner, and the only thing from IRC and you guys is “OHNOES DON’T MESS WITH OUR GLASS HOUSE!!!!”, then I have to ask why all the crying. This stuff comes wth no warranty, and its your choice to assist or not.
And for those who say that windows is painless to upgrade, then you too haven’t really done many “upgrades”, as user customization and fiddling causes just as much ‘pain’ over there as you are whining about here.
Sign me fed up with this drivel.
My 2 cents, yer mileage may vary.
I think the blog post is somewhat on the money.
The first step is informing the users, helping them make good choices, being proactive in giving them realistic expectations so as to avoid disappointment and steering them to community resources that may be useful before they jump off the deep end and get stuck.
People who are saying that software 3rd party repositories are a reality are correct. Some of them are likely to be harmless (it’d be hard to imagine Skype seriously breaking anything) and some are more dangerous (such as Beryl?).
I would imagine that most third party repositories also feature their own communities. A health checker running before an upgrade might advise to check such communities for any reports of conflicts/problems prior to embarking on an upgrade.
Perhaps “unhealthy” might be an unnecessarily perjorative term though. Perhaps someone can come up with another word or phrase to describe a system that diverges from the the ‘normal’ Ubuntu setup and therefore simply will not have been tested to the same degree.
Don’t forget to check for software in /usr/local and /opt
That might break also.
@Deniss Kaarsenmaakers
Sorry about the profanity. But you post really did get on my nervers.
It is very hard to believe any of common customization would make X not start up.
I mean seriously, how is having some codecs in /usr/lib/w32codecs going to make X not start up?
How is even having beryl installed going to be a problem? Its just a python program which you do not have to load. Most beryl users just add beryl-manager to the gnome-session. Each user on a system can choose to use metacity instead of beryl (and visa versa)
XGL, when installed, again, is something a user can choose during login.
Again, I find it hard to believe that because some extra gdm-desktop-file exists, the normal ‘default’ ubuntu sessions stop working.
If in any way this would affect the update of the core ubuntu programs and libraries than the update-script IS TO BLAME.
Is it too much to ask if the update-script will downgrade all packages to an officially ubuntu support version when possible?
Nothing in the PLF or seveas repo’s should give you any problem when upgrading. Nothing at http://www.getdeb.org should give you any problem.
On the other hand having debian repositories in your sources.list :: that should give you problems.
Dennis: please don’t tell people that anything in Seveas repo will cause harm during update. Updates may fail, but it won’t be because of w32codecs. You are not telling the truth.
Lastly: if Ubuntu gives its users less freedom than a normal operating system, i.e.
- you are not allowed to install 3rd party software meant to work on Ubuntu (.deb specifically for your ubuntu version)
- you are not allowed to install software from an repository specially meant to work with your ubuntu verion
If this is the case: THEN WINDOWS GIVES ITS USER MORE FREEDOM.
Then again, I do not think this is the case. And any person claiming that a repo like the PLF of Seveas or having manually installed realplayer, is going to cause any problems is technically completely ignorant.
Three typical arguments:
they change start up scripts: no they don’t
they install different versions of a cerain liberaries: that’s fine, you can have several versions installed at the same time.
the code is monitored/checked by ubuntu-quality-control: muhahaha
Again, the question, how in the world is having glChess or maniaDrive or vDrift being installed in /opt going to be a problem for the updater?
How is having some codecs installed in /usr/lib/w32codecs a problem?
How is having an xmms-wma plugin installed in /usr/lib/xmms going to be a problem?
Seriously, I demand a real technical answer why the common customizations would fail on upgrade!
[b]Dennis, when you ask: “How Healthy is your Ubuntu system?” you sound like a doctor. Which type of doctor are you? The type that says we have wear a rock around our neck or the scientist-type?[/b]
Really, you can not claim that 3rd party ubuntu=specific repos and packages will break an update without even giving one good argument what that would be so…..
There is no profanity in this post this time: no sensor please.
Dennis,
Excellent summary of where the problems lie.
I strongly support a spec for Feisty which will identify problem 3rd party packages and yank them before upgrade.
I think telling people to not dice with Automatix etc is a difficult one. People want the proprietary codecs etc etc which Ubuntu cannot touch for legal reasons. Telling them to avoid it and deal only with backports is fine if you have a sysadmin attitude. It isn’t really if you have a Windows desktop hacker mentality and just want to watch that video or whatever.
Perhaps the developers could put significant pressure on the Automatix and Easybuntu people to include a clean comprehensive uninstall script linked to the package uninstall. The Ubuntu upgrader could then uninstall these packages with the effect of removing everything problematic.
If the writers of these packagers do not want to cooperate then a strong WARNING could be placed somewhere as part of the upgrade instructions.
[...] Alcuni hanno individuato la fonte dei maggiori problemi nei tool non ufficiali di configurazione automatica come Automatix e Easy Ubuntu, ma è facile rendersi conto che la portata dei malfunzionamenti non è limitata solo a questi script. [...]
Martijn: shiver….. :)
Paul: If you can come up with an alternative - less scary - wording, please let me know.
Meneer R: As I am Seveas, I know that it can break! And please read the rest of the comments: I’m not blaming users, but the ones writing broken crap like automatix. And I’m not censoring anything but profanity, rest assured :) I welcome comments such as yours, they clearly show that I didn’t write down my words good enough since you didn’t seem to understand them. I don’t wan’t to take away users freedom to use what they want. I just want Ubuntu to be able to detect ‘bad stuff’ and warn users. I don’t consider w32codecs bad stuff. I *do* consider the early beryl packages bad stuff since they broke things. I hope this example helps, feel free to comment further here or via mail.
Why doesn’t Ubuntu set up package pinning and dependencies properly so that ubuntu-desktop (and similar packages) is never removed? Why doesn’t Ubuntu provide a simple GUI tool that will accept 3rd party apt repository line and automatically set up the required repository priorities so that Ubuntu repositories take preference? Should I start on this?
Dennis: “Unsupported” might be a good enough word, though I suppose “supported” already has a defined meaning as there are official “unsupported” packages. (Humourously “uncanonical” might also be an appropriate word which has a secondary meaning for Ubuntu).
Perhaps “Unrecognised”?
“Ubuntu update has detected unrecognised software on your system. Software from third parties may cause problems when upgrading your Ubuntu operating system and have not been tested by us.
We recommend checking the support resources for the third party packages to see if the upgrade is safe or removing the third party packages prior to the upgrade.
[List of packages]
[Upgrade Now] [Remove Third Party Packages]”
Maybe some Ubuntu people could test some common third party apps (Skype?) and whitelist them?
The second button should probably allow people to select individual packages to remove, rather than just getting rid of all of them.
Have to say, I have used automatix for quite a few packages and my upgrade went flawlessly as far as I can tell
[...] Dennis talked about the issue of unhealthy Ubuntu systems and how they interrupt between distribution upgrades. Dennis presents a list of signs to identify a possibly unhealthy system. He writes: What makes an Ubuntu system unhealthy, you’d say - well here are a few examples: [...]