FastDFS

今天和一位开发的同事吃饭聊到fastdfs,有几个项目一直在用mongoDB做文件存储,但是可能太浪费空间了,可能更适合用FastDFS来替代。

http://code.google.com/p/fastdfs/

FastDFS is an open source high performance distributed file system (DFS). It's major functions include: file storing, file syncing and file accessing, and design for high capacity and load balancing.

FastDFS is an open source high performance distributed file system. It's major functions include: file storing, file syncing and file accessing (file uploading and file downloading), and it can resolve the high capacity and load balancing problem. FastDFS should meet the requirement of the website whose service based on files such as photo sharing site and vidio sharing site.

FastDFS has two roles: tracker and storage. The tracker takes charge of scheduling and load balancing for file access. The storage store files and it's function is file management including: file storing, file syncing, providing file access interface. It also manage the meta data which are attributes representing as key value pair of the file. For example: width=1024, the key is "width" and the value is "1024".

The tracker and storage contain one or more servers. The servers in the tracker or storage cluster can be added to or removed from the cluster by any time without affecting the online services. The servers in the tracker cluster are peer to peer.

The storarge servers organizing by the file volume/group to obtain high capacity. The storage system contains one or more volumes whose files are independent among these volumes. The capacity of the whole storage system equals to the sum of all volumes' capacity. A file volume contains one or more storage servers whose files are same among these servers. The servers in a file volume backup each other, and all these servers are load balancing. When adding a storage server to a volume, files already existing in this volume are replicated to this new server automatically, and when this replication done, system will switch this server online to providing storage services. When the whole storage capacity is insufficiency, you can add one or more volumes to expand the storage capacity. To do this, you need to add one or more storage servers.

The identification of a file is composed of two parts: the volume name and the file name.

主要要测试一下是否适合跨IDC的环境。譬如大文件存储,如何高效的复制到其他IDC,是块复制还是文件级的复制。

时间: 2025-01-19 06:44:51

FastDFS的相关文章

nginx+fastdfs报错conf file must have item

问题描述 nginx+fastdfs报错conf file must have item fastdfs安装配置没问题,使用fastdfs_test测试也能上传,但是安装完nginx及其module后启动nginx时 nginx错误显示:[2015-12-12 18:13:19] ERROR - file: ../storage/trunk_mgr/trunk_shared.c, line: 106, conf file must have item "store_path1"! 而浏

C#和.NET中如何利用FastDFS打造分布式文件系统

背景 海量存储.系统负载的迁移.服务器吞吐的瓶颈等等 让文件系统独立于业务系统 提高整个项目的扩展性以及可维护性 目前主流的方案 MFS FASTDFS GFS LUSTRE HADOOP等等 我选择的是FASTDFS 用一句广告语来说 "免费.快速.找得到".FASTDFS的作者是淘宝的资深架构师余庆,很诙谐.很有爱!!!其他方案还没玩过 暂不评论. 简介 FastDFS是一款开源的轻量级分布式文件系统纯C实现,支持Linux.FreeBSD等UNIX系统类google FS,不是通

linux-Linux搭建Fastdfs文件服务器安装nginx问题

问题描述 Linux搭建Fastdfs文件服务器安装nginx问题 /home/jumper/fastdfs-nginx-module/src//ngx_http_fastdfs_module.c In file included from /home/jumper/fastdfs-nginx-module/src//ngx_http_fastdfs_module.c:6:0: /home/jumper/fastdfs-nginx-module/src//common.c:43:2: error

为什么fastdfs开启防盗链,图片有效,视频不需要token仍然可以播放或下载?

问题描述 为什么fastdfs开启防盗链,图片有效,视频不需要token仍然可以播放或下载? if use token to anti-steal default value is false (0) http.anti_steal.check_token=true token TTL (time to live), seconds default value is 600 http.anti_steal.token_ttl=900 secret key to generate anti-ste

Dubbo分布式架构实战--FastDFS分布式文件系统的安装与使用(单节点)

跟踪服务器:192.168.4.121 (edu-dfs-tracker-01) 存储服务器:192.168.4.125 (edu-dfs-storage-01) 环境:CentOS 6.6 用户:root                                                    数据目录:/fastdfs   (注:数据目录按你的数据盘挂载路径而定) 安装包: FastDFS v5.05 libfastcommon-master.zip(是从FastDFS和Fast

centos-CentOS安装FastDfs错误

问题描述 CentOS安装FastDfs错误 最近在CentOS上安装FastDFS在 编译时出现 /usr/local/lib/libfastcommon.so: undefined reference to sqrt' /usr/local/lib/libfastcommon.so: undefined reference tocos' 错误,请哪位大神帮忙解决一下. 详细错误如下: cc -Wall -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -g -O -D

【JAVA秒会技术之图片上传】基于Nginx及FastDFS,完成图片的上传及展示

基于Nginx及FastDFS,完成商品图片的上传及展示 一.传统图片存储及展示方式      存在问题: 1)大并发量上传访问图片时,需要对web应用做负载均衡,但是会存在图片共享问题 2)web应用服务器的存储空间有限,它是web应用服务器,而不是存储服务器. 3)web应用服务器的本身的io读写性能不高,图片上传下载时,速度偏慢 4)web应用服务器访问图片时,由于图片内容较大,并发量大的时候,会占用web应用服务器的带宽,这样该web应用服务器的其他功能就会受到较大的影响.    解决方

FastDFS文件系统(二) fastdfs和其他文件系统区别

FastDFS文件系统(二) fastdfs和其他文件系统区别 一.概述 普通存储方案:Rsync.DAS(IDE/SATA/SAS/SCSI等块).NAS(NFS.CIFS.SAMBA等文件系统).SAN(FibreChannel, iSCSI, FoE存储网络块),Openfiler.FreeNas(ZFS快照复制)由于生产环境中往往由于对存储数据量很大,而SAN存储价格又比较昂贵,因此大多会选择分布式 存储来解决一下问题: 海量数据存储问题 数据高可用问题(冗余备份)问题 较高的读写性能和

FastDFS(1):Centos7 安装FastDFS+nginx module

本文原文连接: http://blog.csdn.net/freewebsys/article/details/45150941 转载请注明出处! 1,关于FastDFS 摘自:http://www.oschina.net/p/fastdfs FastDFS是一个开源的分布式文件系统,她对文件进行管理,功能包括:文件存储.文件同步.文件访问(文件上传.文件下载)等,解决了大容量存储和负载均衡的问题.特别适合以文件为载体的在线服务,如相册网站.视频网站等等. FastDFS服务端有两个角色:跟踪器

fastdfs-在linux系统中安装FastDFS,运行./make.sh报错

问题描述 在linux系统中安装FastDFS,运行./make.sh报错 我是按照这篇博客来安装的http://www.bkjia.com/Linux/1048217.html 第一步安装libevent成功,后面解压fastdFS(版本跟博客说的不一致),解压后按照博文说的去配置 vim make.sh ,配置好后在执行 "./make.sh C_INCLUDE_PATH=/home/users/wangyanpei/libevent-2.0.21/include LIBRARY_PATH=