I'm working on a slightly bigger project : pros and cons

More general comments, including Off Topic.

Moderator: Pablo

louigi600
Member
Posts: 22
Joined: Thu Feb 05, 2004 9:47 am
Contact:

I'm working on a slightly bigger project : pros and cons

Post by louigi600 » Thu Feb 05, 2004 10:42 am

I'm building a fully featured minimal distribution (based on slackware) that is supposed to fit in a 128Mb compact flash (and currently I have managed to do so excluding mplayer in around 104Mb leaving enough space for the jurnaling database, mplayer and it's extra codecs ).
There is an old version (flash linux) without X which fits in a 64Mb CF leaving 15Mb of free space avalible at http://sourceforge.net/projects/cm3s/ .
As soon as possible I'll place the new version with X for downloading.


:D Pros:
real root filesystem (= your changes get saved with no extra effort)
package management (install and remove native slackware packages)
no busybox (= real init with standard /etc/rc.d initialization, real sh ... etc)
full networking (telnet, ftp, ssh, scp, wget, ping, traceroute, ifconfig, route ... etc)
nearly a full XFree86 with configuration tool and fvwm95
easy to upgrade stuff including kernel
all your ram will be avalible for system
you can use jurnaling filesystems if you like (by default I use ext3)
the compressed root image is 44Mb which means that you can probably boot from a 64Mb CF with a bit of extra effort

:( Cons:
you will need over 4 times as much space than other minimal systems
a bad system crash could corrupt your root filesystem
ram filesystems will perform better than any CF

Some changes have got into the first idea (see talk on this thed below)
I abandoned the original idea of live root fs as in favour of compressed initrd and tgz extra packages + save and reload changes on shutdown and boot.
In this way I can confortably fit my setup in a 64Mb CF.

A firts solid step of this project is avalibe
http://sourceforge.net/projects/bclash/
http://freshmeat.net/projects/clash/
Last edited by louigi600 on Sun Apr 04, 2004 12:54 pm, edited 1 time in total.

dhartman
Member
Posts: 30
Joined: Sat Dec 13, 2003 9:05 pm

Post by dhartman » Thu Feb 05, 2004 6:18 pm

You might check into jffs instead of ext3. jffs (or it might be jffs2 now) is opimized for flash media. You'll want to make the parts of the system that are read-only transfer into ramdisks to speed things up. (I used Midori Linux in the past which operated in this way). For the most part only /etc and your /home directory contain items which need to be written to. My advice is to have your scratch space in ram, then save that to your CF on shutdown. You do not want excessive writes to the CF or you will severely shorten the life. Also mount it with the 'noatime' option.

I'm interested in your progress. I will eventually finish a similar project for my in-car pc project (currently housed on a 20GB laptop drive).

Darrick

lmatter
Contributor
Posts: 632
Joined: Sat Aug 23, 2003 5:13 pm
Location: California, USA

Post by lmatter » Sat Feb 07, 2004 4:33 am

Hey louigi600,

Nice to see you here. Hope we can share some ideas. Since the EPIA M's have the PXE network booting built in, I like taking advange of that.

The next version of minimyth (0.5) will most likely be a more complete distribution and boot over NFS. The current one is filling up fast (up to about 40M). I would like to make a more full distro so will need to run over NFS.

Larry

louigi600
Member
Posts: 22
Joined: Thu Feb 05, 2004 9:47 am
Contact:

Good to see that there are people intrested

Post by louigi600 » Mon Feb 09, 2004 11:07 am

Derrick
Yes in part it would be good to only have part of the root fs writable ... but /var /usr /opt get written on when you start fiddling with packages.
Since 90% of disck usage is attributed to usr how would you deal with that ?

It may then be a good idea to have everithing in ramdisk and save everything on shutdown ?
This could have a negative impact on kernel updating process ?

Larry
I'll be glad to share and exchange ideas ... but beware I'm not that experienced so I guess I may benefit more from your knowlage than you may benefit from mine!!!!!
In any case I'll ge glad to show my ignorance to learn from my mistakes !

dhartman
Member
Posts: 30
Joined: Sat Dec 13, 2003 9:05 pm

Post by dhartman » Mon Feb 09, 2004 1:11 pm

louigi600,

Check out the midori linux project. It's way out dated in many regards, but the way it utilized the compact flash was and still is about the best I've seen. They used 'packedcramfs' which utilized multiple ramdisks. In many ways what we're trying to accomplish is much easier than Midori (we don't need a cross compiler--its target was a Transmeta Crusoe powered device).

/var should be in ramfs. If you insist on using package management similar to Slackware, I suggest a scratch partition or 'chroot' type target on a host machine, then a separate script to create the CF image.
/usr and /opt should only need to be writeable when new packages are added. At that time, re-write the entire CF.
Many settings in /etc even don't need to be written to when the system is live. Symbolic links are your friend. Put the changing stuff in var and a symbolic link in /etc pointing toward that changing value.

Look at the compact flash as a target instead of as a normal system. Think embedded instead of normal full-blown operation.

You can also significantly reduce size on alot of the final binaries by stripping debugging symbols out after everything is working. (see LFS documentation for how to do it--I can look it up if you want.).

Good luck!
Darrick

louigi600
Member
Posts: 22
Joined: Thu Feb 05, 2004 9:47 am
Contact:

Post by louigi600 » Tue Feb 10, 2004 9:57 am

Derrick ...
I'll keep in mind all the intresting stuff you just said and see hou I'll manage to emproove my project.

I'm still a bit puzzeled on how unexperienced people could manage a kernel upgrade on non live FS.

Package management is not the only thing that writes to /var (logging is on /var/log and many processes write in /var/run) ... I guess that var is called so because it's a varying part of FS ?

Changing the the way the package management tools work (for saving the stuff) would mean changing the original slackware pkgtool stripts ?
Does anyone know what is the mean time before faliure (MTBF) of CF devices and what is the expected avarage rewrite count before failure ?

Slackware has all it's binaries already stripped and anything I compile gets stripped once I'm over with any debugging.

louigi600
Member
Posts: 22
Joined: Thu Feb 05, 2004 9:47 am
Contact:

Post by louigi600 » Fri Mar 26, 2004 10:25 am

I've done some work to squeeze it into a 64Mb CF using compressed root image.

I need some help to boot up now ?

Any sugestions ?

David

lmatter
Contributor
Posts: 632
Joined: Sat Aug 23, 2003 5:13 pm
Location: California, USA

Post by lmatter » Sat Mar 27, 2004 1:23 am

David,

How are you trying to boot? Syslinux? Grub? And what are the errors you are getting.

Which kernel are you using?

Larry

louigi600
Member
Posts: 22
Joined: Thu Feb 05, 2004 9:47 am
Contact:

Post by louigi600 » Sun Mar 28, 2004 3:19 pm

First of all I solved the probles ...

Just incase anyone is finding the same trouble this is what was happening:

Setup:
Slackware based minmal system with kernel 2.4.24 (best I could patch for hostAP)
Lilo boot loader
6Mb compressed initrd (uncopressed : used 15Mb fof a total of 32Mb)
/usr on tmpfs (96Mb with 24k inode ) tgz'd to 16Mb (but no X for now)

{ this setup with no X fits in a 32Mb CF leaving around 7 Mb free for saving changes on shutdown .... and we are talking about real glibc stuff with no busybox around along with native slackware package manager.
once I get the save on shutdown working You can easyly install the other stuff you might need.
I guess that a system with minimal X and fvwm would fit into a 64Mb CF no problem }

Problem: annoying errors on fsck in rc.S and mount says / is on /dev/hda (but I'm booting to a 32Mb ramdisk) ....

Solution:
Skipped all fsck on ram related FS
removed a stupid old mtab in /etc that got there from the minimal system I use to generate many of my minimal systems.

louigi600
Member
Posts: 22
Joined: Thu Feb 05, 2004 9:47 am
Contact:

Post by louigi600 » Thu Apr 01, 2004 3:35 pm

Ok the first solid step in this project is online
http://sourceforge.net/project/showfile ... _id=105712
http://freshmeat.net/projects/clash/

Open to any comments

louigi600
Member
Posts: 22
Joined: Thu Feb 05, 2004 9:47 am
Contact:

Post by louigi600 » Sun Apr 04, 2004 12:44 pm

Beta 2 will hopefully be avalible withins a few hours.
I have added :
syslinux booting (so that no separate lilo installing is necessary if you prefer)
loading and saving changes at boot and shutdown respectively.

If any volonteers test this out and tell me if it works for them too it would be great ;-)

David

cyrus007
Member
Posts: 28
Joined: Fri May 21, 2004 8:22 pm
Location: Dallas, TX
Contact:

Checkout FREEPIA

Post by cyrus007 » Fri May 21, 2004 9:51 pm

Hi Luigi,
Have you checked out FREEPIA project at http://www.freepia.org. They use Freevo as their base system instead of MythTV but the install CD has the option to run it off the CD or install on various media like CF, HD, USB etc. The latest beta version is being developed on the Gentoo development environment. You might want to look at it to get ideas.

HTH
Cyrus

louigi600
Member
Posts: 22
Joined: Thu Feb 05, 2004 9:47 am
Contact:

Post by louigi600 » Thu Aug 26, 2004 4:33 pm

I've added a tool that lets you build a custom distribution from scratch (BYOD) crafted specially for clash.

It produces a system that has most busybox commands along with common networking tools (ssh, scp, sshd, telnetd, ftpd, inetd, iptables and dhcp tools) no X yet. It fits in less than 7 Mb uncompressed. This makes it possible to target the system on a 4Mb CF device. I guess this is good point to start from if you need more specific stuff of the CF devices that are avalible nowdays.

Hopefully I'll add this new release (clash1.0-rc3)to freshmeat tonight .

Have fun

louigi600
Member
Posts: 22
Joined: Thu Feb 05, 2004 9:47 am
Contact:

Post by louigi600 » Tue Sep 21, 2004 7:42 am

The project has reached a stable developement status and is now at version 1.2.
I've recenrly added wireless tools to BYOD.

Have fun.
Regards
David

louigi600
Member
Posts: 22
Joined: Thu Feb 05, 2004 9:47 am
Contact:

Post by louigi600 » Thu Oct 21, 2004 2:00 pm

Version 2.0 of clash is close to it's release and will include some intresting features for the "Build Your Own Distro" like:
dialog driven package selection,
tiny web server,
php scripting,
tiny db server and more....

It will be avalible asap on fresmeat and sourceforge.
Have fun
David

Post Reply