基础介绍
Regular Expressions, BNF and RPN
编译原理的基础介绍有很多。简洁精要,需要功力。
参考:A2 Computing
正则
正则引擎的对比:regex perf
吐槽正则的: Regular Expressions: Now You Have Two Problems
于我心有戚戚焉。不过《精通正则表达式》还是要看的,看完就知道某些高级技巧走火入魔,更想用BNF,哈哈!
REBOL的PARSE
rebol的parse是另一种实现风格
业精于勤荒于嬉,行成于思毁于随
Regular Expressions, BNF and RPN
编译原理的基础介绍有很多。简洁精要,需要功力。
参考:A2 Computing
正则引擎的对比:regex perf
吐槽正则的: Regular Expressions: Now You Have Two Problems
于我心有戚戚焉。不过《精通正则表达式》还是要看的,看完就知道某些高级技巧走火入魔,更想用BNF,哈哈!
rebol的parse是另一种实现风格
以debian为例
apt-get install python-certbot-apache
certbot --apache
certbot --apache certonly
证书信息将保存在/etc/letsencrypt目录下
certbot renew --quiet
例如添加某个域名 abc.xxx.com
certbot certonly --webroot -w /var/www/abc/ -d abc.xxx.com
对应的apache配置文件示例如下:
<VirtualHost abc.xxx.com:80>
ServerName abc.xxx.com
DocumentRoot "/var/www/abc"
<Directory "/var/www/abc">
</Directory>
</VirtualHost>
<IfModule mod_ssl.c>
<VirtualHost abc.xxx.com:443>
ServerName abc.xxx.com
DocumentRoot "/var/www/abc"
<Directory "/var/www/abc">
</Directory>
SSLCertificateFile /etc/letsencrypt/live/abc.xxx.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/abc.xxx.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>
假设目录为 /var/www/mojo,站点为 mojo.xxx.com
.
├── main.pl
└── static
├── css
│ └── style.css
├── js
├── view
└── .well-known
5 directories, 2 files
注意,main.pl为主文件。
通过Mojolicious::Static将static目录下的静态文件加入 / 路径,效果例如 http(s)://mojo.xxx.com/css/style.css 访问到style.css 样式表
view 目录下存放 Mojo::Template 的模板文件,例如 some.ep
.well-known 用于 letsencrypt 的证书处理
#!/usr/bin/perl
use strict;
use warnings;
use FindBin;
use JSON;
use Mojo::Template;
use Mojolicious::Lite;
use Mojolicious::Static;
my $static = app->static();
push @{ $static->paths }, "$FindBin::RealBin/static";
get '/' => sub {
my $self = shift;
$self->render( template => 'index', format => 'html', handler => 'ep' );
};
post '/do_task' => sub {
my $self = shift;
my $text = $self->param('text');
$self->render( text => "<pre>$text</pre>" );
};
app->start;
__DATA__
@@ index.html.ep
<html>
<head>
<meta http-equiv=Content-Type content="text/html;charset=utf-8">
<link rel="stylesheet" href="css/style.css" />
<title>...</title>
</head>
<body>
...
</body>
</html>
注意443的配置是在certbot成功生成证书之后添加的
<VirtualHost mojo.xxx.com:80>
ServerName mojo.xxx.com
DocumentRoot "/var/www/mojo"
<Directory "/var/www/mojo">
</Directory>
<Location />
SetHandler perl-script
PerlHandler Plack::Handler::Apache2
PerlSetVar psgi_app /var/www/mojo/main.pl
</Location>
</VirtualHost>
<IfModule mod_ssl.c>
<VirtualHost mojo.xxx.com:443>
ServerName mojo.xxx.com
DocumentRoot "/var/www/mojo"
SSLCertificateFile /etc/letsencrypt/live/mojo.xxx.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/mojo.xxx.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
Options Indexes Includes FollowSymLinks Multiviews
<Directory "/var/www/mojo">
</Directory>
<Location />
SetHandler perl-script
PerlHandler Plack::Handler::Apache2
PerlSetVar psgi_app /var/www/mojo/main.pl
</Location>
</VirtualHost>
</IfModule>
注意这边目录写成 /var/www/mojo/static,因为我们在main.pl里指定了静态文件路径是static子目录
而letsencrypt需要检查的.well-known子目录在static下面
certbot certonly --webroot -w /var/www/mojo/static -d mojo.xxx.com
如果写成 /var/www/mojo则会失败,原因是如果.well-known直接放在/var/www/mojo下,main.pl中并未设置route到该目录,因此certbot提示找不到路径,renew失败
示例letsencrypt证书更新配置文件(自动生成的)
/etc/letsencrypt/renewal/mojo.xxx.com.conf
# renew_before_expiry = 30 days
version = 0.10.2
archive_dir = /etc/letsencrypt/archive/mojo.xxx.com
cert = /etc/letsencrypt/live/mojo.xxx.com/cert.pem
privkey = /etc/letsencrypt/live/mojo.xxx.com/privkey.pem
chain = /etc/letsencrypt/live/mojo.xxx.com/chain.pem
fullchain = /etc/letsencrypt/live/mojo.xxx.com/fullchain.pem
# Options used in the renewal process
[renewalparams]
authenticator = webroot
installer = None
account = 6b0e486cbbe399702b1600fd60ea646a
webroot_path = /var/www/mojo/static,
[[webroot_map]]
mojo.xxx.com = /var/www/mojo/static
假设远程vps为remote,ip地址为xxx.xxx.xxx.xxx,用户名为someusr,密码为somepasswd。
本地提供socks tunnel,假设socks proxy port 8888。
本地提供http proxy,再转发到socks tunnel,假设http proxy port 9999。
本机浏览器、Dropbox等客户端直接使用socks tunnel。
android / ios 手机配置wlan连接使用该http proxy。
下载 plink。
根据用户名、密码登录: plink -C -D 8888 -N someusr@xxx.xxx.xxx.xxx -pw somepasswd
安装 privoxy
# config.txt
listen-address 0.0.0.0:9999
forward-socks5 / 127.0.0.1:8888 .
ssh someusr@remote -N -D 127.0.0.1:8888 -F ~/.ssh/config
安装 polipo
# /etc/polipo/config
proxyAddress = "0.0.0.0"
proxyPort = 9999
socksParentProxy = "127.0.0.1:8888"
socksProxyType = socks5
直接执行sudo polipo
即可成功开启本地http proxy
假设本地开启proxy的机器内网ip为 192.168.1.111
curl -x http://192.168.1.111:9999 https://ipinfo.io
curl -x socks5h://192.168.1.111:8888 https://ipinfo.io
firefox扩展 foxyproxy
var direct = 'DIRECT';
var http_proxy = 'SOCKS5 127.0.0.1:8888; DIRECT';
var tunnel_list = [
"gmail.com",
"google.com",
"google.com.hk",
"googleapis.com"
];
function FindProxyForURL(url, host) {
if(! host) return direct;
for (var i = 0; i < tunnel_list.length; i += 1) {
var v = tunnel_list[i];
if ( dnsDomainIs(host, v)) {
return http_proxy;
}
}
return direct;
};
var direct = 'DIRECT';
var http_proxy = 'SOCKS5 127.0.0.1:8888; DIRECT';
var white_list = [
".cn",
".com.cn",
".qq.com",
".jd.com",
".360buyimg.com",
".baidu.com",
".bdstatic.com",
".douban.com",
"weibo.com",
".taobao.com",
".alipay.com",
".alicdn.com",
".taobaocdn.com"
];
var black_list = [
"google.com",
"youtube.com"
];
function FindProxyForURL(url, host) {
if(! host) return direct;
for (var i = 0; i < black_list.length; i += 1) {
var v = black_list[i];
var dotv = '.' + v;
if ( dnsDomainIs(host, dotv) || dnsDomainIs(host, v)) {
return http_proxy;
}
}
for (var i = 0; i < white_list.length; i += 1) {
var v = white_list[i];
var dotv = '.' + v;
if ( dnsDomainIs(host, dotv) || dnsDomainIs(host, v)) {
return direct;
}
}
return http_proxy;
};
ssh xxx.xxx.xxx.xxx -l someusr
需要预先安装 sshpass
sshpass -p somepasswd ssh someusr@xxx.xxx.xxx.xxx
本地用ssh-keygen -t rsa生成 remote_rsa/remote_rsa.pub的密钥对。
把本地生成的remote_rsa.pub传到remote上的/home/someusr/.ssh/目录下,并改名为authorized_keys。
本地使用ssh -i remote_rsa someusr@xxx.xxx.xxx.xxx
命令进行远程登录。
本地~/.ssh/config
配置私钥,后续可以用 ssh someusr@remote 直接登陆
注意remote_rsa是私钥
Host remote
HostName xxx.xxx.xxx.xxx
Port 22
User someusr
IdentityFile ~/.ssh/remote_rsa
批量指定多台机器免登陆
Host *.xxx.com
IdentityFile ~/.ssh/remote_rsa
User someusr
假设登陆顺序: 本地 -> middle -> remote
假设middle登陆端口为12345,remote登陆端口为23456
在三台机器上安装netcat
本地配置 ~/.ssh/config,使得本地能够不输密码,通过middle登陆remote
Host middle
User miduser
Hostname mid.xxx.com
Port 12345
PreferredAuthentications publickey
IdentityFile ~/.ssh/middle_rsa
Host remote
Hostname rem.yyy.com
User remuser
ProxyCommand ssh -q middle nc -q0 %h 23456
IdentityFile ~/.ssh/remote_rsa
middle的 ~/.ssh/authorized_keys 为 ~/.ssh/middle_rsa 所对应的公钥
remote的 ~/.ssh/authorized_keys 为 ~/.ssh/remote_rsa 所对应的公钥
rsync使用不受影响: rsync -avzu root@remote:/root/test/ /root/test
ansible需要配置 /etc/ansible/hosts,例如
[multihop]
rem.yyy.com
[multihop:vars]
ansible_ssh_common_args = -o ForwardAgent=yes -o ControlMaster=auto -o ControlPersist=300s -o ServerAliveInterval=60 -o ProxyCommand="ssh -q middle.xxx.com nc -q0 %h 23456"
参考 Keep Your Linux SSH Session From Disconnecting,在~/.ssh/config
中添加
Host *
ServerAliveInterval 60
rsync --progress -avzu --delete remote_usr@remote.xxx.com:/var/remote/ /var/local
sshpass -p "remote_passwd" rsync --progress -avzu --delete -e ssh remote_usr@remote.xxx.com:/var/remote/ /var/local
假设搭建3proxy服务的server ip为xxx.xxx.xxx.xxx
准备配置proxy用户名为somepu
,密码为somepw
开启的http proxy端口为2356
,开启的socks5 proxy端口为3456
示例/etc/3proxy/3proxy.cfg
文件
#nserver 8.8.8.8
daemon
service
users somepu:CL:somepw
log /var/log/3proxy.log D
logformat "- +_L%t.%. %N.%p %E %U %C:%c %R:%r %O %I %h %T"
rotate 7
auth strong
flush
allow somepu
maxconn 20
# starting HTTP proxy with disabled NTLM auth ( -n )
proxy -p2356 -n
# starting SOCKS proxy
socks -p3456 -n
# starting POP3 proxy on port 110
#pop3p
启动3proxy: cd /etc/3proxy && ./3proxy 3proxy.cfg
用curl通过http代理取页面:curl -x somepu:somepw@xxx.xxx.xxx.xxx:2356 http://abbypan.blogspot.com -v
用curl通过socks5代理取页面:curl --proxy socks5h://somepu:somepw@xxx.xxx.xxx.xxx:3456 http://abbypan.blogspot.com -v
apt install trojan
准备公私钥对,申请证书,例如letsencrypt。
server/client可以分别申请,相互信任。
假设密码为mypasswd
假设server端:
配置/usr/local/etc/trojan/config.json
:
{
"run_type": "server",
"local_addr": "0.0.0.0",
"local_port": 443,
"remote_addr": "127.0.0.1",
"remote_port": 80,
"password": [
"mypasswd"
],
"log_level": 1,
"ssl": {
"cert": "/home/someusr/.cert/fullchain.pem",
"key": "/home/someusr/.cert/privkey.pem",
"key_password": "",
"cipher": "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305",
"cipher_tls13": "TLS_AES_128_GCM_SHA256:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_256_GCM_SHA384",
"prefer_server_cipher": true,
"alpn": [
"http/1.2",
"http/1.3",
"h2"
],
"reuse_session": true,
"session_ticket": false,
"session_timeout": 600,
"plain_http_response": "",
"curves": "",
"dhparam": ""
},
"tcp": {
"prefer_ipv4": false,
"no_delay": true,
"keep_alive": true,
"reuse_port": false,
"fast_open": false,
"fast_open_qlen": 20
},
"mysql": {
"enabled": false,
"server_addr": "127.0.0.1",
"server_port": 3306,
"database": "trojan",
"username": "trojan",
"password": "",
"key": "",
"cert": "",
"ca": ""
}
}
启动
systemctl start trojan
添加开机启动
systemctl enable trojan
假设client端:
/home/someclient/share/trojan
,/home/someclient/share/trojan/fullchain.pem
/home/someclient/share/trojan/privkey.pem
配置/home/someclient/share/trojan/config.json
:
{
"run_type": "client",
"local_addr": "127.0.0.1",
"local_port": 8888,
"remote_addr": "xxx.example.com",
"remote_port": 443,
"password": [
"mypasswd"
],
"log_level": 1,
"ssl": {
"cert": "fullchain.pem",
"key": "privkey.pem",
"key_password": "",
"cipher": "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305",
"cipher_tls13": "TLS_AES_128_GCM_SHA256:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_256_GCM_SHA384",
"prefer_server_cipher": true,
"alpn": [
"http/1.2",
"http/1.3",
"h2"
],
"reuse_session": true,
"session_ticket": false,
"session_timeout": 600,
"plain_http_response": "",
"curves": "",
"dhparam": ""
},
"tcp": {
"prefer_ipv4": false,
"no_delay": true,
"keep_alive": true,
"reuse_port": false,
"fast_open": false,
"fast_open_qlen": 20
},
"mysql": {
"enabled": false,
"server_addr": "127.0.0.1",
"server_port": 3306,
"database": "trojan",
"username": "trojan",
"password": "",
"key": "",
"cert": "",
"ca": ""
}
}
启动
cd /home/someclient/share/trojan
trojan -c config.json
apt install v2ray
准备公私钥对,申请证书,例如letsencrypt。
假设server端:
配置/usr/local/etc/v2ray/config.json
:
{
"log": {
"loglevel": "info"
},
"inbounds": [
{
"port": 443,
"protocol": "vless",
"settings": {
"clients": [
{
"id": "66666666-6666-6666-6666-666666666666",
"flow": "xtls-rprx-origin",
"level": 0
}
],
"decryption": "none",
"fallbacks": [
{
"dest": 80
}
]
},
"streamSettings": {
"network": "tcp",
"security": "xtls",
"xtlsSettings": {
"alpn": [
"http/1.2"
],
"certificates": [
{
"certificateFile": "/home/someusr/.cert/fullchain.pem",
"keyFile": "/home/someusr/.cert/privkey.pem"
}
]
}
}
}
],
"outbounds": [
{
"protocol": "freedom"
}
]
}
启动
systemctl start v2ray
添加开机启动
systemctl enable v2ray
假设client端:
/home/someclient/share/v2ray
,配置/home/someclient/share/v2ray/config.json
:
{
"log": {
"access": "",
"error": "",
"loglevel": "warning"
},
"inbounds": [
{
"port": 8888,
"listen": "127.0.0.1",
"protocol": "socks",
"sniffing": {
"enabled": true,
"destOverride": [
"http",
"tls"
]
},
"settings": {
"auth": "noauth",
"udp": true,
"ip": null,
"clients": null
},
"streamSettings": null
}
],
"outbounds": [
{
"tag": "proxy",
"protocol": "vless",
"settings": {
"vnext": [
{
"address": "xxx.example.com",
"port": 443,
"users": [
{
"id": "66666666-6666-6666-6666-666666666666",
"flow": "xtls-rprx-origin",
"level": 0,
"encryption": "none"
}
]
}
],
"servers": null,
"response": null
},
"streamSettings": {
"network": "tcp",
"security": "",
"tlsSettings": null,
"tcpSettings": null,
"kcpSettings": null,
"wsSettings": null,
"httpSettings": null,
"quicSettings": null
},
"mux": {
"enabled": true
}
},
{
"tag": "direct",
"protocol": "freedom",
"settings": {
"vnext": null,
"servers": null,
"response": null
},
"streamSettings": null,
"mux": null
},
{
"tag": "block",
"protocol": "blackhole",
"settings": {
"vnext": null,
"servers": null,
"response": {
"type": "http"
}
},
"streamSettings": null,
"mux": null
}
],
"dns": null,
"routing": {
"domainStrategy": "IPIfNonMatch",
"rules": []
}
}
启动
cd /home/someclient/share/v2ray
v2ray run
准备公私钥对,申请证书,例如letsencrypt。
假设:
config.json
{
"tunnel_path": "/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/",
"server_settings": {
"certfile": "/home/someusr/.cert/fullchain.pem",
"keyfile": "/home/someusr/.cert/privkey.pem",
"forward_addr": "http://127.0.0.1:80",
"listen_host": "0.0.0.0",
"listen_port": 443
},
"client_settings": {
"server_host": "xxx.xxx.xxx.xxx",
"server_port": 443,
"server_domain": "xxx.example.com",
"listen_host": "127.0.0.1",
"listen_port": 8888
}
}
假设config.json存放路径为/etc/overtls/config.json
启动:
overtls -r server -c config.json
编辑 /etc/systemd/system/overtls.service
:
[Unit]
Description=overtls
Documentation=overtls
After=network.target network-online.target nss-lookup.target
[Service]
Type=simple
StandardError=journal
ExecStart="overtls" -r server -c "/etc/overtls/config.json"
ExecReload=/bin/kill -HUP $MAINPID
LimitNOFILE=51200
Restart=on-failure
RestartSec=1s
[Install]
WantedBy=multi-user.target
启动:
systemctl daemon-reload
systemctl enable overtls.service
systemctl start overtls.service
overtls -r client -c config.json
以曾斯琪FX视频为例: 720878-China-Zeng-Siqi-FX
这个网站比较有意思,没有用f4m链接,而是用rtmp视频流。
swf有好几个,反编译看的头晕。
不爽之下决定直接上wireshark,小样,看你还藏!
先打开wireshark设置抓包,然后在浏览器中观看视频,视频开始播放之后,停止抓包
在wireshark抓包中找到视频连接、播放的指令:
因此,此视频的RTMP信息为:
rtmp://96.7.248.111/ondemand?_fcs_vhost=cp76677.edgefcs.net
mp4:s/lyYmQyZjo3BEcsOVWgTlDdnVgcnuhXjG/DOcJ-FxaFrRg4gtDEwOmk2OjBrO6qGv_
rtmpdump下载地址:http://rtmpdump.mplayerhq.hu/
下载flv视频,保存为zsq.flv :
rtmpdump -r
"rtmp://96.7.248.111/ondemand?_fcs_vhost=cp76677.edgefcs.net" -y
"mp4:s/lyYmQyZjo3BEcsOVWgTlDdnVgcnuhXjG/DOcJ-FxaFrRg4gtDEwOmk2OjBrO6qGv_"
-o zsq.flv
tshark 命令行抓包,-i 4 指定网卡序号:
tshark -i 4 -w a.cap
tshark 从a.cap提取rtmp连接url:
tshark -r "a.cap" -Y "rtmpt"|grep connect
tshark 从a.cap提取播放路径:
tshark -r "a.cap" -Y "rtmpt"|grep play
吐槽:
这边rtmp视频流协议的关键字是”rtmpt”,之前直接用”rtmp”死活过滤不出来。
原因看这里:http://www.wireshark.org/docs/dfref/#section_r
这就叫网络协议太多,4个字缩写都不够用了。
二十四年春,穆叔如晋。
范宣子逆之,问焉,曰:“古人有言曰,‘死而不朽’,何谓也?”
穆叔未对。
宣子曰:“昔匄之祖,自虞以上,为陶唐氏,在夏为御龙氏,在商为豕韦氏,在周为唐、杜氏,晋主夏盟为范氏,其是之谓乎?”
穆叔曰:“以豹所闻,此之谓世禄,非不朽也。 鲁有先大夫曰臧文仲,既没,其言立。 其是之谓乎?豹闻之,太上有立德,其次有立功,其次有立言,虽久不废,此之谓不朽。 若夫保姓受氏,以守宗祊,世不绝祀,无国无之。 禄之大者,不可谓不朽。”