Nginx简易文件上传

1. 编译nginx,添加Nginx Upload Module

下载nginx所依赖软件包、nginx源码及nginx-upload-module(nginx-http-auth-digest 很久不再更新,有兼容问题,不推荐):

mkdir nginx_upload && cd nginx_upload
wget https://www.openssl.org/source/openssl-1.0.2j.tar.gz
wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.39.tar.gz
wget https://zlib.net/fossils/zlib-1.2.8.tar.gz

#wget https://github.com/atomx/nginx-http-auth-digest/archive/v1.0.0.tar.gz
#mv v1.0.0.tar.gz nginx_http_auth_digest_1.0.0.tar.gz

wget http://nginx.org/download/nginx-1.10.2.tar.gz
git clone -b 2.255 https://github.com/vkholodkov/nginx-upload-module

编译安装:

tar xvf openssl-1.0.2j.tar.gz
tar xvf pcre-8.39.tar.gz
tar xvf zlib-1.2.8.tar.gz
#tar xvf nginx_http_auth_digest_1.0.0.tar.gz
tar xvf nginx-1.10.2.tar.gz
cd nginx-1.10.2

#./configure --prefix=/home/linhan/local --add-module=../nginx-upload-module --add-module=../nginx-http-auth-digest-1.0.0 --with-openssl=../openssl-1.0.2j --with-http_gzip_static_module --with-http_ssl_module --with-pcre=../pcre-8.39 --with-zlib=../zlib-1.2.8
./configure --prefix=/home/linhan/local --add-module=../nginx-upload-module --with-openssl=../openssl-1.0.2j --with-http_gzip_static_module --with-http_ssl_module --with-pcre=../pcre-8.39 --with-zlib=../zlib-1.2.8

make && make install

如无错误,则nginx已经被安装在/home/linhan/local/sbin下。其配置文件则是/home/linhan/local/conf/nginx.conf

2. https支持

加密浏览器与服务器之间的传输,更安全。

2.1 生成私钥和签名证书

openssl req -new -newkey rsa:2048 -sha256 -nodes -out example.csr -keyout example.key -subj "/C=CN/ST=ShenZhen/L=ShenZhen/O=Example Inc./OU=Web Security/CN=example.com"

生成 csr 文件后,提供给 CA 机构,签署成功后,就会得到一個 example.crt 证书文件。
不需要CA机构签署,则可以手工签署:

openssl x509 -req -days 3650 -in example.csr -signkey example.key -out example.crt

或者干脆一步到位:

openssl req -x509 -nodes -days 10000 -newkey rsa:1024 -keyout example.key -out example.pem

2.2 nginx配置文件修改

文后附上完整配置文件,先看个大概:

server {
    ......
    listen 8000 ssl;
    server_name localhost;

    #证书文件
    ssl_certificate /home/linhan/local/crt/lanhin.crt;
    #私钥文件
    ssl_certificate_key /home/linhan/local/crt/lanhin.key;

    ssl_session_cache shared:SSL:1m;
    ssl_session_timeout 5m;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers HIGH:!aNULL:!MD5;

    ssl_prefer_server_ciphers on;
    ......
}

3. 用户名+密码控制访问

采用这种密码控制访问的方法,用同一浏览器(似乎)只有在第一次访问时需要输入密码,验证通过后续访问不再需要输入密码,尚未找到更好且简洁的方法。

3.1 生成密码文件

echo -n 'sony:' >> ~/local/conf/.htpasswd
openssl passwd -apr1 >> ~/local/conf/.htpasswd

执行第二条命令时会两次要求输入要设置的密码。检查一下密码文件:

cat ~/local/conf/.htpasswd
sony:$apr1$d7QT7LBL$jLxcgh6xFhNNUNnru6xjy/

3.2 配置修改

参考下面location /的最后两行:

server {
    listen 80 default_server;
    listen [::]:80 default_server ipv6only=on;

    root /usr/share/nginx/html;
    index index.html index.htm;

    server_name localhost;

    location / {
        try_files $uri $uri/ =404;
        auth_basic "Restricted Content";  # 访问控制
        auth_basic_user_file /etc/nginx/.htpasswd;  # 密码文件
    }
}

4. 启动服务器

cd /tmp && mkdir 0 1 2 3 4 5 6 7 8 9
~/local/sbin/nginx -c ~/local/conf/nginx.conf

5. 先来看看效果

6. 完整配置与后端处理程序代码

# File: nginx.conf
worker_processes 1;

events {
    worker_connections 1024;
}

http {
    include mime.types;
    default_type application/octet-stream;

    sendfile on;

    keepalive_timeout 65;

    server {
     client_max_body_size 100m;
        listen 8000 ssl;
        server_name localhost;

 #证书文件
 ssl_certificate /home/linhan/local/crt/example.crt;
 #私钥文件
 ssl_certificate_key /home/linhan/local/crt/example.key;

 ssl_session_cache shared:SSL:1m;
 ssl_session_timeout 5m;

 ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
 ssl_ciphers HIGH:!aNULL:!MD5;

 ssl_prefer_server_ciphers on;

 # Upload form should be submitted to this location
 location /upload {
     # Pass altered request body to this location
     upload_pass @test;

     # Store files to this directory
     # The directory is hashed, subdirectories 0 1 2 3 4 5 6 7 8 9 should exist
     upload_store /tmp 1;

     # Allow uploaded files to be read only by user
     upload_store_access all:rw;

     # Set specified fields in request body
     upload_set_form_field $upload_field_name.name "$upload_file_name";
     upload_set_form_field $upload_field_name.content_type "$upload_content_type";
     upload_set_form_field $upload_field_name.path "$upload_tmp_path";

     # Inform backend about hash and size of a file
     upload_aggregate_form_field "$upload_field_name.md5" "$upload_file_md5";
     upload_aggregate_form_field "$upload_field_name.size" "$upload_file_size";

     upload_pass_form_field "^submit$|^description$";

     upload_cleanup 400 404 499 500-505;
 }

 # Pass altered request body to a backend
 location @test {
     proxy_pass http://localhost:8080;
 }
    }
}
# File: index.html
<!DOCTYPE html>
<html>
<head>
<title>File upload demo!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>File upload demo!</h1>

<form name="upload" method="POST" enctype="multipart/form-data" action="/upload">
      <input type="file" name="file"><br>
      <input type="file" name="file"><br>
      <input type="submit" name="submit" value="Upload">
    </form>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
# File: upload_daemon.py
#!/usr/bin/env python
# encoding: utf-8

import tornado.ioloop
import tornado.web
import tornado.log
import tornado.httpserver
from tornado.options import define, options
import logging
import tornado.gen
import tornado.process
import os
import os.path
import magic
import mimetypes

define('debug', type=bool, default=False, help="enable debug, default False")
define('host', type=str, default="127.0.0.1", help="http listen host, default 127.0.0.1")
define('port', type=int, default=8080, help="http listen port, default 8080")
define('storage_path', type=str, default="storage", help="file storage path")
options.parse_command_line()

logger = logging.getLogger('fileserver')

project_dir_path = os.path.abspath(os.path.dirname(__file__))

if not os.path.exists(options.storage_path):
    os.mkdir(options.storage_path)


class LinuxUtils(object):

    @staticmethod
    @tornado.gen.coroutine
    def mv(src, dest):
        cmd = ["mv", src, dest]
        proc = tornado.process.Subprocess(cmd)
        ret = yield proc.wait_for_exit(raise_error=False)
        raise tornado.gen.Return(ret == 0)

    @staticmethod
    @tornado.gen.coroutine
    def rm(file):
        cmd = ["rm", file]
        proc = tornado.process.Subprocess(cmd)
        ret = yield proc.wait_for_exit(raise_error=False)
        raise tornado.gen.Return(ret == 0)



class UploadHandler(tornado.web.RequestHandler):

    @tornado.gen.coroutine
    def post(self):
        keys = self.request.arguments.keys()
        if "file.path" not in keys:
            self.set_status(status_code=400, reason="file field not exist.")
            self.write("400: file field not exist.")
            return
        files = list()
        file_path = self.request.arguments['file.path']
        for index in xrange(len(file_path)):
            file = {}
            file['name'] = self.request.arguments['file.name'][index]
            file['content_type'] = self.request.arguments['file.content_type'][index]
            file['path'] = self.request.arguments['file.path'][index]
            file['md5'] = self.request.arguments['file.md5'][index]
            file['size'] = self.request.arguments['file.size'][index]
            files.append(file)
        # mv tmp file to save store storage
        for file in files:
            # debug
            src_file = file['path']
            print "src_file:", src_file
            mime = magic.from_file(src_file, mime=True)
            ext = mimetypes.guess_extension(mime, False)
            print "ext:", ext
            dest_file = os.path.join(
                options.storage_path,
                file['md5'] + "_" + file['name']
            )
            print "dest_file:",dest_file
            if not os.path.exists(dest_file):
                yield LinuxUtils.mv(
                    src_file,
                    dest_file
                )
            else:
                yield LinuxUtils.rm(
                    src_file
                )
            if ext:
                file['path'] = file['md5'] + ext
            else:
                file['path'] = file['md5']
        self.write({"data": files})


class Application(tornado.web.Application):

    def __init__(self):
        handlers = [
            (r'/upload', UploadHandler),
        ]

        settings = dict()
        settings['debug'] = True
        super(Application, self).__init__(handlers, **settings)

if __name__ == '__main__':
    application = Application()
    http_server = tornado.httpserver.HTTPServer(application)
    http_server.listen(
        options.port,
        address=options.host
    )
    logger.info("http server listen on %s:%d", options.host, options.port)
    tornado.ioloop.IOLoop.current().start()

7. 参考文献

发表评论

电子邮件地址不会被公开。 必填项已用*标注