背景
nas开启nfs存储共享，默认情况下只能给IP/24做限制, 达不到安全效果
需要增加kerberos策略校验，并且持久化kerberos数据，避免容器重启丢失数据
环境描述
宿主机系统：CentOS Linux release 7.9.2009 (Core)
Docker版本：Docker version 20.10.6, build 370c289
一、容器部署kerberos
# 注意添加 /etc/hosts
192.168.10.10 kerberosclient.wo.com
192.168.10.10 kerberosserver.wo.com1.kerberos配置文件
mkdir -p /data/kerberos/dockerfile
cd /data/kerberos/dockerfile@FQDN@ 后面使用sed更改为大写域名 = WO.COM
@fqdn@ 后面使用sed更改为小写域名 = wo.com
@kdc_server@ kdc服务器地址 = kerberosclient.wo.com
krb5.conf
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
[libdefaults]
dns_lookup_realm = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
rdns = false
pkinit_anchors = FILE:/etc/pki/tls/certs/ca-bundle.crt
default_realm = @FQDN@
default_ccache_name = FILE:/tmp/krb5cc_cli_%{uid}
[realms]
@FQDN@ = {
kdc = @kdc_server@
admin_server = @kdc_server@
admin_keytab = /var/lib/krb5kdc/admin.keytab
database_name = /var/lib/krb5kdc/principal
}
[domain_realm]
.@fqdn@ = @FQDN@kadm5.acl
*/admin@@FQDN@ *kerberos.sh
#!/bin/bash
fqdn="wo.com"
FQDN="WO.COM"
kdc_server="kerberosserver.wo.com"
PASS="Fa1Q@D@N"
KRB5_KTNAME="/var/lib/krb5kdc/admin.keytab"
inited="/var/lib/krb5kdc/inited"
sed -i "s#@kdc_server@#${kdc_server}#g" /etc/krb5kdc/kdc.conf
sed -i "s#@fqdn@#${fqdn}#g" /etc/krb5kdc/kdc.conf
sed -i "s#@FQDN@#${FQDN}#g" /etc/krb5kdc/kdc.conf
sed -i "s#@FQDN@#${FQDN}#g" /etc/krb5kdc/kadm5.acl
function init_user() {
if [ -f "${inited}" ];then
# TODO没找到指定该文件的配置项, 将该文件还原到/etc/目录
cp /var/lib/krb5kdc/.k5.${FQDN} /etc/krb5kdc/.k5.${FQDN}
echo "kerberos已存在, 跳过初始化"
return;
fi
echo "begin init user"
# create kerberos database
echo -e "${PASS}\n${PASS}" | kdb5_util create -s
# create admin
echo -e "${PASS}\n${PASS}" | kadmin.local -q "addprinc root/admin"
kadmin.local -q "ktadd -k /var/lib/krb5kdc/admin.keytab root/admin"
# create client
echo -e "${PASS}\n${PASS}" | kadmin.local -q "addprinc -randkey nfs/kerberosclient.${fqdn}"
# create client keytab
kadmin.local -q "ktadd -norandkey -k ${KRB5_KTNAME} nfs/kerberosclient.${fqdn}"
kadmin.local -q "xst -k /app/cert/krb5.keytab -norandkey nfs/kerberosclient.${fqdn}"
# client使用
cp /etc/krb5kdc/kdc.conf /app/cert/krb5.conf
# 将该文件持久化存储
cp /etc/krb5kdc/.k5.${FQDN} /var/lib/krb5kdc/.k5.${FQDN}
touch "${inited}"
echo "user inite success"
}
function main() {
init_user
/usr/local/bin/supervisord -n -c /etc/supervisord.conf
}
mainsupervisord.conf
[supervisord]
logfile=/var/log/supervisord/supervisord.log ; supervisord log file
logfile_maxbytes=50MB ; maximum size of logfile before rotation
logfile_backups=10 ; number of backed up logfiles
loglevel=error ; info, debug, warn, trace
pidfile=/var/run/supervisord.pid ; pidfile location
nodaemon=false ; run supervisord as a daemon
minfds=1024 ; number of startup file descriptors
minprocs=200 ; number of process descriptors
user=root ; default user
childlogdir=/var/log/supervisord/ ; where child log files will live
[program:krb5-kdc]
command=service krb5-kdc start
autostart=true
autorestart=true
[program:krb5-admin-server]
command=service krb5-admin-server start
autostart=true
autorestart=true
[supervisorctl]dockerfile
FROM ubuntu:xenial
ENV DEBIAN_FRONTEND noninteractive
RUN sed -i s@/archive.ubuntu.com/@/mirrors.aliyun.com/@g /etc/apt/sources.list \
&& sed -i s@/security.ubuntu.com/@/mirrors.aliyun.com/@g /etc/apt/sources.list \
&& apt update \
&& apt install -y python-dev python-pip python-wheel python-setuptools python-pkg-resources krb5-admin-server krb5-kdc \
&& rm -rf /var/lib/apt/lists/* \
&& mkdir -p /var/log/supervisord /app/cert \
&& pip install supervisor==4.2.4
COPY krb5.conf /etc/krb5kdc/kdc.conf
COPY kadm5.acl /etc/krb5kdc/kadm5.acl
COPY krb5.conf /etc/krb5.conf
COPY kerberos.sh /app/kerberos.sh
COPY supervisord.conf /etc/supervisord.conf
WORKDIR /app
CMD ["/bin/bash", "/app/kerberos.sh"]2.构建镜像
docker build -t kerberos:1.0.0 .3.运行镜像
mkdir -p /data/kerberos/data
cd /data/kerberosstart.sh
#!/bin/bash
docker rm -f kerberos
# 持久化数据, 避免容器重启数据库丢失
# /app/cert 用于给client的keytab和conf配置
# /var/lib/krb5kdc 数据库文件存放路径
# /etc/krb5kdc/.k5.xxx master文件, 也需要持久化
# 在kerberos.sh脚本时, 会将.k5文件放到/var/lib/krb5kdc
docker run -itd \
-p 88:88 \
-p 749:749 \
-v /data/kerberos/data/cert:/app/cert \
-v /data/kerberos/data/db:/var/lib/krb5kdc \
--name=kerberos \
kerberos:1.0.0二、nas配置krb5.keytab
/data/kerberos/data/cert/krb5.keytab将krb5.keytab上传到nas
# 开启krb5校验
三、nfs客户端机器
1.安装krb5
yum -y install krb5-workstation ufs-utils
文件从kerberos服务端获取，拷贝到客户端(注意区分机器)
cp /data/kerberos/data/cert/krb5.conf /etc/krb5.conf
cp /data/kerberos/data/cert/krb5.keytab /etc/krb5.keytab
客户端启动rpc-gssd
systemctl restart rpc-gssd2.验证
kinit -kt /etc/krb5.keytab nfs/kerberosclient.wo.com@WO.COM
klist
挂载
mount -o vers=4,sec=krb5 kerberosclient.wo.com:/volume1/data /mnt
四、参考文档
如何配置 NFS 共享文件夹以使用 Kerberos？ - Synology 知识中心
Synology NAS NFS Kerberos 配置与使用 – 个人笔记分享
NFS | DSM - Synology 知识中心
使用Docker快速搭建Kerberos环境 - 知乎
基于Kerberos认证的NFS服务器_nfs kerberos_黑色蒲G英~的博客-CSDN博客
五、其他报错信息
1.用户和组显示nobody
用户和组显示nobody
更改/etc/idmapd.conf 将Domain 改为fqdn 的域名
systemctl restart rpcidmapd
2.创建文件提示权限不足
在nas上,上传kerberos密钥对的配置，增加ID映射，对应用那个user
3.access denied by server while mounting
# 查看kerberos应用日志
tail -f /var/log/k*4.mount.nfs an incorrect mount option was specified
# 没有krb5.conf krb5.keytab 2个文件
# 然后启动rpc-gssd
systemctl restart rpc-gssd5.挂载后文件都显示777权限
改为无映射
