Week 8: Distributed File Systems: chubby and parallel file systems
- Lecture notes:
-
Distributed File Systems - Lecture notes
-
Distributed File Systems - Lecture slides
(6 per page)
- Supplemental notes:
-
- Chubby
-
Mike Burrows,
The Chubby Lock Service for Loosely-Coupled Distributed Systems, Google Inc.,
OSDI'06: Seventh Symposium on Operating System Design and Implementation,
Seattle, WA, November, 2006.
(pdf)
The definitive paper on Google's Chubby lock service.
- Google FS (GFS)
-
Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung,
The Google File System ,
19th ACM Symposium on Operating Systems Principles, October, 2003.
- Hadoop Distributed File System (HDFS)
-
HDFS main page, hadoop.apache.org
HDFS Architecture Guide
hadoop.apache.org
Hadoop Architecture video by Keyth McDonald,
IBM DB2 WLM Team. (under 15 minutes long)
- Dropbox
-
- Kevin Modzelewski, How We've Scaled Dropbox,
Computer Science Colloquium, Stanford Center for Professional Development.
- Other stuff
-
Frank Schmuck and Roger Haskin,
GPFS: A Shared-Disk File System for Large Computing Clusters
.
Proceedings of the FAST 2002 Conference on File and Storage Technologies,
IBM Almaden Research Center San Jose, CA
-
xFS: Serverless Network File System,
University of California, Berkeley
-
Google Megastore 3 Billion Writes And 20 Billion Read Transactions Daily, highscalability.com.
-
Google Spanner's Most Surprising Revelation: NoSQL Is Out And NewSQL Is In
highscalability.com.
- Buzzwords:
-
Chubby
lock manager, name server, Chubby cell, replica, named locks, notification,
Paxos
-
GFS
Chunkserver, master, chunk handle, opeartion log, chunk lease, chunk replication, data flow, control flow
-
HDFS
NameNode, DataNode, blocks, Transaction log, FsImage
-
Dropbox
Blockserver, metaserver, notification server, load balancer