The Library 6 * ten thousands problem 
Posted by SebastienBailard
|
The Library 6 * ten thousands problem January 25, 2010 08:38PM |
These are my notes on scaling issues I don't understand yet.
[objects.reprap.org]
The Library 6 * ten thousands problem:
How do we keep track of
10,000 files
for
10,000 "projects"
by
10,000 user-developers, artists, 3D scanner hobbyists, museum curators, architectural historians, engineering students, etc.
on
10,000 pages
in
10,000 languages
for
10,000 years.
Note: these numbers are approximate, but will guide our thinking on this problem until we have a solution. I think there may be 10,000 ways. And only a few good ones. And probably, 200-400 years, minimum.
We don't want to spend 10,000 hours figuring this out, so it would be best to enlist some experts. We also don't want to keep parts on 10,000 damn websites. 2-3 websites is better than 10,000 websites, and 1 website twice or thrice as good, because it means 10,000 less outgoing links to mirrored content to check.
Seriously?
Yes. 200-400+ years for things like scans of sculpture from the 1600s.
10,000 user-developers, minimum.
At least a few dozen languages.
Definitely 1000 gears, etc.
Certainly a few dozen repstraps.
Hopefully a few dozen RepRaps.
Only one Policy that we generally try to follow, and the responsible site administrators definitely follow, and that needs editing before we show it to the EFF and Archive.org for review, let alone a museum curator or paper library archivist!
This is utterly impossible!
No it's not. We can do it with clever scripting, lots of server space, and an "agile" website written with good software architecture and social networking principals in mind.
We might be able to do it all with a mediawiki. Maybe.
Are you crazy?
Nope. We're patient. And unstoppable. [link to library forum] Not that we don't mind volunteer help from software architects, library archivists, and users who want to upload to RepRap.org. In fact, we are actively soliciting volunteer help from software architects, library archivists, and users uploads. --Sebastien Bailard 02:58, 24 January 2010 (UTC) ([email protected])
Why bother?
Many reasons. And hey, wikipedia is a good helpful resource for people, and RepRap machines are about making fun stuff and useful stuff for people, so the RepRap.org should be a good, maximally helpful resource for people scanning with RepRaps, priting with RepRaps and building RepRaps. Aka RepRap user-developers.
Does this matter? I only came here to make a RepRap, I don't care about that stuff
No worries! Our primary job is the RepRap documentation, but many of us work on how to document things to make with RepRaps. And why are you reading this? This is a library issue.
Notes, Theory
We are still defining the problem. "Projects, Parts, Arts, and Docs" is pithy, but I don't know what it means.
This is an information science problem, and a social networking problem.
Retrieved text ends, snapshot of page 2009 Jan 25.
Triffid hunter posted this query on the discussion page:
perhaps archive.org would extend their archive to cover objects? they have already taken the leap into seeing digital information as something to archive under their charter.
Me:
Yup, they're good people and 'projects, parts, arts, and docs' fall under their charter. I want them to be the secondary mirror of everything on RepRap.org, rather than for-profit websites-of-the-month. But RepRap.org doesn't need more outgoing links right now.
On RepRap.org, it's our role to host, curate, steward, and administrate the RepRap community's instructions for projects, parts and their metadata: verbal and computer instructions (files), photoessays, and community/energy (chat, eyeballs) etc. perfectly and uniquely bonded and organized. Or RepRap.org as instructions and as users, starts to evaporate into a frustrating mess of outgoing links. Again. (Exhibit A: [blog.reprap.org])
Also, they don't exploit strong social networking principles like youtube or Zach-the-RRRF-guy's site do. We need to operate using strong social networking principles in order to motivate contributions. Community motivates contribution, and contribution motivates community. By accidentally forking contributions, Zach et. al. have accidentally forked the community. And vice versa. Cleaning that up is our job.
I do want Archive.org contributing to our Policy and helping us with software and server infrastructure because they're great people who do good work.
Very much best to discuss it in Library Administration, Announcements, and Policy and keep our notes on Library/6x10000s Problem. Otherwise, the conversation starts to fork. So please start a new thread quoting this, and we can thoughtfully hash it out in the open. Because that's what we do with Library Policy matters. (and maybe wipe the "discussion" page, replacing it with a link to your new "Library/6x10000s Problem" thread, once you create it.)--Sebastien Bailard 22:41, 24 January 2010 (UTC)
[objects.reprap.org]
The Library 6 * ten thousands problem:
How do we keep track of
10,000 files
for
10,000 "projects"
by
10,000 user-developers, artists, 3D scanner hobbyists, museum curators, architectural historians, engineering students, etc.
on
10,000 pages
in
10,000 languages
for
10,000 years.
Note: these numbers are approximate, but will guide our thinking on this problem until we have a solution. I think there may be 10,000 ways. And only a few good ones. And probably, 200-400 years, minimum.
We don't want to spend 10,000 hours figuring this out, so it would be best to enlist some experts. We also don't want to keep parts on 10,000 damn websites. 2-3 websites is better than 10,000 websites, and 1 website twice or thrice as good, because it means 10,000 less outgoing links to mirrored content to check.
Seriously?
Yes. 200-400+ years for things like scans of sculpture from the 1600s.
10,000 user-developers, minimum.
At least a few dozen languages.
Definitely 1000 gears, etc.
Certainly a few dozen repstraps.
Hopefully a few dozen RepRaps.
Only one Policy that we generally try to follow, and the responsible site administrators definitely follow, and that needs editing before we show it to the EFF and Archive.org for review, let alone a museum curator or paper library archivist!
This is utterly impossible!
No it's not. We can do it with clever scripting, lots of server space, and an "agile" website written with good software architecture and social networking principals in mind.
We might be able to do it all with a mediawiki. Maybe.
Are you crazy?
Nope. We're patient. And unstoppable. [link to library forum] Not that we don't mind volunteer help from software architects, library archivists, and users who want to upload to RepRap.org. In fact, we are actively soliciting volunteer help from software architects, library archivists, and users uploads. --Sebastien Bailard 02:58, 24 January 2010 (UTC) ([email protected])
Why bother?
Many reasons. And hey, wikipedia is a good helpful resource for people, and RepRap machines are about making fun stuff and useful stuff for people, so the RepRap.org should be a good, maximally helpful resource for people scanning with RepRaps, priting with RepRaps and building RepRaps. Aka RepRap user-developers.
Does this matter? I only came here to make a RepRap, I don't care about that stuff
No worries! Our primary job is the RepRap documentation, but many of us work on how to document things to make with RepRaps. And why are you reading this? This is a library issue.
Notes, Theory
We are still defining the problem. "Projects, Parts, Arts, and Docs" is pithy, but I don't know what it means.
This is an information science problem, and a social networking problem.
Retrieved text ends, snapshot of page 2009 Jan 25.
Triffid hunter posted this query on the discussion page:
perhaps archive.org would extend their archive to cover objects? they have already taken the leap into seeing digital information as something to archive under their charter.
Me:
Yup, they're good people and 'projects, parts, arts, and docs' fall under their charter. I want them to be the secondary mirror of everything on RepRap.org, rather than for-profit websites-of-the-month. But RepRap.org doesn't need more outgoing links right now.
On RepRap.org, it's our role to host, curate, steward, and administrate the RepRap community's instructions for projects, parts and their metadata: verbal and computer instructions (files), photoessays, and community/energy (chat, eyeballs) etc. perfectly and uniquely bonded and organized. Or RepRap.org as instructions and as users, starts to evaporate into a frustrating mess of outgoing links. Again. (Exhibit A: [blog.reprap.org])
Also, they don't exploit strong social networking principles like youtube or Zach-the-RRRF-guy's site do. We need to operate using strong social networking principles in order to motivate contributions. Community motivates contribution, and contribution motivates community. By accidentally forking contributions, Zach et. al. have accidentally forked the community. And vice versa. Cleaning that up is our job.
I do want Archive.org contributing to our Policy and helping us with software and server infrastructure because they're great people who do good work.
Very much best to discuss it in Library Administration, Announcements, and Policy and keep our notes on Library/6x10000s Problem. Otherwise, the conversation starts to fork. So please start a new thread quoting this, and we can thoughtfully hash it out in the open. Because that's what we do with Library Policy matters. (and maybe wipe the "discussion" page, replacing it with a link to your new "Library/6x10000s Problem" thread, once you create it.)--Sebastien Bailard 22:41, 24 January 2010 (UTC)
|
Re: The Library 6 * ten thousands problem January 25, 2010 08:42PM |
I've moved this to the forum.
This is the sort of thing we have to worry about as RepRap librarians, if we're doing more than just hosting Mendel.
This is the sort of thing we have to worry about as RepRap librarians, if we're doing more than just hosting Mendel.
Sorry, only registered users may post in this forum.