代码样例-Socks代理
本文档包含编程请求Socks代理服务器的代码样例,供开发者参考。
代码样例使用说明
- 代码样例不能直接运行,因为代码中的订单SecretId
o1fjh1re9o28876h7c08
、signaturexxxxx
、代理服务器59.38.241.25:23918
、用户名username
、密码password
都是虚构的,请替换成您自己的信息。 - 代码样例正常运行所需的运行环境和注意事项在样例末尾均有说明,使用前请仔细阅读。
- 使用代码样例过程中遇到问题请联系售后客服,我们会为您提供技术支持。
特别注意
以下代码样例均为基础样例,运行基础样例并不能保证成功爬取目标网站。目标网站通常具备反爬机制,比如跳转需要输入验证码的页面。
我们建议您在开发过程中基于基础样例进行如下改进:
- 添加IP池管理;
- 合理控制对目标网站的请求频率,建议对同一网站1个代理IP每秒请求不超过1次;
- 发出的http请求尽可能带上完整的header信息。
Python3
requests
requests(推荐)
使用提示
- http/https网页均可适用
- 运行环境要求:requests >= 2.10.0
- socks支持是requests的额外特性,需要安装才能使用:
pip install requests[socks]
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
使用requests请求代理服务器
请求http和https网页均适用
"""
import requests
# 提取代理API接口,获取1个代理IP
api_url = "https://dps.kdlapi.com/api/getdps/?secret_id=o1fjh1re9o28876h7c08&signature=xxxxx&num=1&pt=2&sep=1"
# 获取API接口返回的代理IP
proxy_ip = requests.get(api_url).text
# 用户名密码认证(私密代理/独享代理)
username = "username"
password = "password"
proxies = {
"http": "socks5h://%(user)s:%(pwd)s@%(proxy)s/" % {"user": username, "pwd": password, "proxy": proxy_ip},
"https": "socks5h://%(user)s:%(pwd)s@%(proxy)s/" % {"user": username, "pwd": password, "proxy": proxy_ip}
}
# 白名单方式(需提前设置白名单)
# proxies = {
# "http": "socks5h://%(proxy)s/" % {"proxy": proxy_ip},
# "https": "socks5h://%(proxy)s/" % {"proxy": proxy_ip}
# }
# 要访问的目标网页
target_url = "https://dev.kdlapi.com/testproxy"
# 使用代理IP发送请求
response = requests.get(target_url, proxies=proxies)
# 获取页面内容
if response.status_code == 200:
print(response.text)
aiohttp
aiohttp
使用提示
- 基于aiohttp的代码样例支持访问http,https网页
- aiohttp不是python原生库,需要安装才能使用:
pip install aiohttp, pip install aiohttp-socks
- aiohttp只支持Python3.5及以上
- 如Windows系统使用aiohttp访问https网站抛出异常,在import asyncio后调用 asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())即可解决。
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
使用aiohttp请求代理服务器
请求http和https网页均适用
"""
import aiohttp
import aiohttp_socks
import asyncio
async def fetch(url, proxy):
# 设置代理连接器
connector = aiohttp_socks.ProxyConnector.from_url(proxy)
async with aiohttp.ClientSession(connector=connector) as session:
async with session.get(url) as response:
print(f"Status: {response.status}")
print(await response.text())
# 用你的SOCKS代理替换此URL
proxy_url = "socks5://user:password@host:port"
# 用你想要访问的实际URL替换此URL
url_to_fetch = "http://example.com"
asyncio.run(fetch(url_to_fetch, proxy_url))`
Python2
requests
requests(推荐)
使用提示
- http/https网页均可适用
- 运行环境要求:requests >= 2.10.0
- socks支持是requests的额外特性,需要安装才能使用:
pip install requests[socks]
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""
使用requests请求代理服务器
请求http和https网页均适用
"""
import requests
# 提取代理API接口,获取1个代理IP
api_url = "https://dps.kdlapi.com/api/getdps/?secret_id=o1fjh1re9o28876h7c08&signature=xxxxx&num=1&pt=2&sep=1"
# 获取API接口返回的代理IP
proxy_ip = requests.get(api_url).text
# 用户名密码认证(私密代理/独享代理)
username = "username"
password = "password"
proxies = {
"http": "socks5h://%(user)s:%(pwd)s@%(proxy)s/" % {"user": username, "pwd": password, "proxy": proxy_ip},
"https": "socks5h://%(user)s:%(pwd)s@%(proxy)s/" % {"user": username, "pwd": password, "proxy": proxy_ip}
}
# 白名单方式(需提前设置白名单)
# proxies = {
# "http": "socks5h://%(proxy)s/" % {"proxy": proxy_ip},
# "https": "socks5h://%(proxy)s/" % {"proxy": proxy_ip}
# }
# 要访问的目标网页
target_url = "https://dev.kdlapi.com/testproxy"
# 使用代理IP发送请求
response = requests.get(target_url, proxies=proxies)
# 获取页面内容
if response.status_code == 200:
print response.text
Python-Selenium
Chrome
Chrome(IP白名单,推荐)
使用提示
- 基于白名单方式使用Selenium+Chrome认证代理
- 运行环境要求python2/3 + selenium + Chrome + Chromedriver + Windows/Linux/macOS
- 下载chromedriver(注意chromedriver版本要和Chrome版本对应)
- selenium不是python原生库,需要安装才能使用:
pip install selenium
(注意:selenium 4.6版本开始,无需手动下载driver) - 请注意替换代码中的部分信息:
${ip:port}:代理IP:端口号,如:"59.38.241.25:23918"
${chromedriver_path}:您本机chromedriver驱动存放路径,如:"C:\chromedriver.exe"
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from selenium import webdriver
import time
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=socks5h://${ip:port}') # 代理IP:端口号
# selenium 4.6及以上
driver = webdriver.Chrome(options=chrome_options)
# ${chromedriver_path}: chromedriver驱动存放路径
# driver = webdriver.Chrome(executable_path="${chromedriver_path}", options=chrome_options)
driver.get("https://dev.kdlapi.com/testproxy")
# 获取页面内容
print(driver.page_source)
# 延迟3秒后关闭当前窗口,如果是最后一个窗口则退出
time.sleep(3)
driver.close()
Chrome(不支持用户名密码认证)
使用提示
- Chrome 本身不支持 SOCKS 认证,因此Selenium也无法支持,请使用 Selenium + PhantomJS进行认证。
PhantomJS
PhantomJS(用户名密码认证)
使用提示
- 基于用户名密码使用 Selenium + PhantomJS认证代理(PhantomJS是无界面浏览器)
- 运行环境要求python2/3 + selenium + PhantomJS + Windows/Linux/macOS
- 点此下载PhantomJS(推荐使用2.1.1版)
- ${executable_path}:您本机PhantomJS驱动存放路径,如:"C:\phantomjs-2.1.1-windows\bin\phantomjs.exe"
#!/usr/bin/env python
# encoding: utf-8
from selenium import webdriver
import time
#先下载phantomjs包文件,再填入phantomjs.exe的路径 (路径不要包含中文)
executable_path = '${executable_path}'
service_args=[
'--proxy=host:port', #此处替换您的代理ip,如59.38.241.25:23918
'--proxy-type=socks5',
'--proxy-auth=username:password' #用户名密码
]
driver=webdriver.PhantomJS(service_args=service_args,executable_path=executable_path)
driver.get('https://dev.kdlapi.com/testproxy')
print(driver.page_source)
time.sleep(3)
driver.close()
Java
okhttp3
okhttp-3.8.1
使用提示
- 此样例同时支持访问http和https网页
- 使用用户名密码验证时必须使用
java.net.Authenticator.setDefault()
方法 - 添加依赖
import okhttp3.*;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.PasswordAuthentication;
import java.net.Proxy;
public class TestProxyOKHttpClient {
public static void main(String args[]) throws IOException {
// 目标网站
String targetUrl = "https://dev.kdlapi.com/testproxy";
// 用户名密码认证(私密代理/独享代理)
final String username = "username";
final String password = "password";
String ip = "tpsxxx.kdlapi.com"; // 代理服务器IP
int port = 20818;
Proxy proxy = new Proxy(Proxy.Type.SOCKS, new InetSocketAddress(ip, port));
java.net.Authenticator.setDefault(new java.net.Authenticator(){//通过设置一个全局的Authenticator,实现socks设置Authenticator用户名密码
private PasswordAuthentication authentication = new PasswordAuthentication(username, password.toCharArray());
@Override
protected PasswordAuthentication getPasswordAuthentication()
{
return authentication;
}
});
OkHttpClient client = new OkHttpClient.Builder()
.proxy(proxy)
.build();
Request request = new Request.Builder()
.url(targetUrl)
.addHeader("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3100.0 Safari/537.36")
.build();
Response response = client.newCall(request).execute();
System.out.println(response.body().string());
}
}
jdk
IP白名单
// 请求socks代理服务器(IP白名单)
// http和https网页均适用
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.HttpURLConnection;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.net.URL;
import java.util.zip.GZIPInputStream;
public class JavaSocksWhitelist {
public static void main(String[] args) throws IOException {
String proxyIp = "59.38.241.25"; // 代理服务器IP
int proxyPort = 23918; // 代理服务器端口
// 要访问的目标网页
String pageUrl = "https://dev.kuaidaili.com/testproxy";
Proxy proxy = new Proxy(Proxy.Type.SOCKS, new InetSocketAddress(proxyIp, proxyPort));
URL url = new URL(pageUrl);
HttpURLConnection httpUrlConnection = (HttpURLConnection) url.openConnection(proxy);
httpUrlConnection.setRequestMethod("GET");
httpUrlConnection.setConnectTimeout(5*1000); // 设置超时时间
httpUrlConnection.setRequestProperty("Accept-Encoding", "gzip"); // 添加gzip压缩让数据传输更块
// 发起请求
httpUrlConnection.connect();
// 输出状态码
System.out.println("code: " + httpUrlConnection.getResponseCode());
// 读取返回内容
InputStream inputStream = httpUrlConnection.getInputStream();
String encoding = httpUrlConnection.getContentEncoding();
// 处理gzip压缩
if (encoding.equals("gzip")) inputStream = new GZIPInputStream(inputStream);
String message = getContentFromInputStream(inputStream);
// 输出返回内容
System.out.println(message);
// 关闭输入流和连接
inputStream.close();
httpUrlConnection.disconnect();
}
// 读取输入流中的内容
public static String getContentFromInputStream(InputStream inputStream) throws IOException {
final int bufferSize = 1024;
final char[] buffer = new char[bufferSize];
final StringBuilder out = new StringBuilder();
Reader in = new InputStreamReader(inputStream, "UTF-8");
for (;;) {
int rsz = in.read(buffer, 0, buffer.length);
if (rsz < 0) break;
out.append(buffer, 0, rsz);
}
return out.toString();
}
}
用户名密码认证
// 请求socks代理服务器(用户名密码认证)
// http和https网页均适用
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.Authenticator;
import java.net.HttpURLConnection;
import java.net.InetSocketAddress;
import java.net.PasswordAuthentication;
import java.net.Proxy;
import java.net.URL;
import java.util.zip.GZIPInputStream;
public class JavaSocksAuth {
public static void main(String[] args) throws IOException {
String proxyIp = "59.38.241.25"; // 代理服务器IP
int proxyPort = 23918; // 代理服务器端口
// 用户名密码认证(私密代理/独享代理)
String username = "username"; // 用户名
String password = "password"; // 密码
// 要访问的目标网页
String pageUrl = "https://dev.kuaidaili.com/testproxy";
//确保使用用户名密码鉴权正常运行
System.setProperty("jdk.http.auth.tunneling.disabledSchemes", "");
Proxy proxy = new Proxy(Proxy.Type.SOCKS, new InetSocketAddress(proxyIp, proxyPort));
URL url = new URL(pageUrl);
HttpURLConnection httpUrlConnection = (HttpURLConnection) url.openConnection(proxy);
httpUrlConnection.setRequestMethod("GET");
httpUrlConnection.setConnectTimeout(5*1000); // 设置超时时间
httpUrlConnection.setRequestProperty("Accept-Encoding", "gzip"); // 添加gzip压缩让数据传输更块
// 设置代理认证
Authenticator authenticator = new Authenticator() {
public PasswordAuthentication getPasswordAuthentication() {
return (new PasswordAuthentication(username,
password.toCharArray()));
}
};
Authenticator.setDefault(authenticator);
// 发起请求
httpUrlConnection.connect();
// 输出状态码
System.out.println("code: " + httpUrlConnection.getResponseCode());
// 读取返回内容
InputStream inputStream = httpUrlConnection.getInputStream();
String encoding = httpUrlConnection.getContentEncoding();
// 处理gzip压缩
if (encoding.equals("gzip")) inputStream = new GZIPInputStream(inputStream);
String message = getContentFromInputStream(inputStream);
// 输出返回内容
System.out.println(message);
// 关闭输入流和连接
inputStream.close();
httpUrlConnection.disconnect();
}
// 读取输入流中的内容
public static String getContentFromInputStream(InputStream inputStream) throws IOException {
final int bufferSize = 1024;
final char[] buffer = new char[bufferSize];
final StringBuilder out = new StringBuilder();
Reader in = new InputStreamReader(inputStream, "UTF-8");
for (;;) {
int rsz = in.read(buffer, 0, buffer.length);
if (rsz < 0) break;
out.append(buffer, 0, rsz);
}
return out.toString();
}
}
GoLang
标准库
用户名密码认证
// 请求socks代理服务器(用户名密码认证)
// http和https网页均适用
package main
import (
"compress/gzip"
"fmt"
"golang.org/x/net/proxy"
"io"
"io/ioutil"
"net/http"
"os"
)
func main() {
// 用户名密码认证(私密代理/独享代理)
username := "username"
password := "password"
auth := proxy.Auth{
User: username,
Password: password,
}
proxy_str := "59.38.241.25:23918"
// 目标网页
page_url := "http://dev.kdlapi.com/testproxy"
// 设置代理
dialer, err := proxy.SOCKS5("tcp", proxy_str, &auth, proxy.Direct)
if err != nil {
fmt.Println(err.Error())
os.Exit(1)
}
// 请求目标网页
client := &http.Client{Transport: &http.Transport{Dial: dialer.Dial}}
req, _ := http.NewRequest("GET", page_url, nil)
req.Header.Add("Accept-Encoding", "gzip") //使用gzip压缩传输数据让访问更快
res, err := client.Do(req)
if err != nil {
// 请求发生异常
fmt.Println(err.Error())
} else {
defer res.Body.Close() //保证最后关闭Body
fmt.Println("status code:", res.StatusCode) // 获取状态码
// 有gzip压缩时,需要解压缩读取返回内容
if res.Header.Get("Content-Encoding") == "gzip" {
reader, _ := gzip.NewReader(res.Body) // gzip解压缩
defer reader.Close()
io.Copy(os.Stdout, reader)
os.Exit(0) // 正常退出
}
// 无gzip压缩, 读取返回内容
body, _ := ioutil.ReadAll(res.Body)
fmt.Println(string(body))
}
}
IP白名单认证
// 请求socks代理服务器(已设置IP白名单)
// http和https网页均适用
package main
import (
"compress/gzip"
"fmt"
"golang.org/x/net/proxy"
"io"
"io/ioutil"
"net/http"
"os"
)
func main() {
// 代理服务器
proxy_str := "59.38.241.25:23918"
// 目标网页
page_url := "http://dev.kdlapi.com/testproxy"
// 设置代理
dialer, err := proxy.SOCKS5("tcp", proxy_str, nil, proxy.Direct)
if err != nil {
fmt.Println(err.Error())
os.Exit(1)
}
// 请求目标网页
client := &http.Client{Transport: &http.Transport{Dial: dialer.Dial}}
req, _ := http.NewRequest("GET", page_url, nil)
req.Header.Add("Accept-Encoding", "gzip") //使用gzip压缩传输数据让访问更快
res, err := client.Do(req)
if err != nil {
// 请求发生异常
fmt.Println(err.Error())
} else {
defer res.Body.Close() //保证最后关闭Body
fmt.Println("status code:", res.StatusCode) // 获取状态码
// 有gzip压缩时,需要解压缩读取返回内容
if res.Header.Get("Content-Encoding") == "gzip" {
reader, _ := gzip.NewReader(res.Body) // gzip解压缩
defer reader.Close()
io.Copy(os.Stdout, reader)
os.Exit(0) // 正常退出
}
// 无gzip压缩, 读取返回内容
body, _ := ioutil.ReadAll(res.Body)
fmt.Println(string(body))
}
}
CSharp
HttpToSocks5
HttpToSocks5Proxy
using System;
using System.Net.Http;
using System.IO;
using System.IO.Compression;
using MihaZupan;
namespace httptosocks
{
class Program
{
static void Main(string[] args)
{
// 目标网页
string page_url = "http://dev.kdlapi.com/testproxy";
// 代理服务器
string proxy_ip = "59.38.241.25";
int proxy_port = 23918;
// 用户名密码认证(私密代理/独享代理)
string username = "username";
string password = "password";
// 设置代理 (私密/独享代理&已添加白名单)
// var proxy = new HttpToSocks5Proxy(new[] {
// new ProxyInfo(proxy_ip, proxy_port),
// });
// 设置Socks5代理 (私密/独享代理&未添加白名单)
var proxy = new HttpToSocks5Proxy(new[] {
new ProxyInfo(proxy_ip, proxy_port, username, password),
});
// 请求目标网页
var handler = new HttpClientHandler { Proxy = proxy };
HttpClient httpClient = new HttpClient(handler, true);
HttpRequestMessage httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, page_url);
httpRequestMessage.Headers.Add("Accept-Encoding", "Gzip"); // 使用gzip压缩传输数据让访问更快
var httpsGet = httpClient.SendAsync(httpRequestMessage);
var result = httpsGet.Result;
Console.WriteLine((int)result.StatusCode); // 获取状态码
// 解压缩读取返回内容
using (StreamReader reader = new StreamReader(new GZipStream(result.Content.ReadAsStreamAsync().Result, CompressionMode.Decompress))) {
Console.WriteLine(reader.ReadToEnd());
}
}
}
}
Node.js
request-promise
request-promise(IP白名单)
使用提示
- http/https网页均可适用
- 请先安装
request-promise
和socks-proxy-agent
:npm install request-promise socks-proxy-agent
// 引入第三方 `socks-proxy-agent` 模块
const SocksProxyAgent = require('socks-proxy-agent');
// 引入第三方 `request-promise`
const rp = require('request-promise');
// 代理服务器ip和端口
const proxy_ip = '59.38.241.25';
const proxy_port = 23918;
// 设置代理
let proxy = `socks5h://${proxy_ip}:${proxy_port}`
let agent = new SocksProxyAgent(proxy);
// 要访问的目标网页
let url = 'http://dev.kuaidaili.com/testproxy';
let options = {
uri: url,
agent: agent,
resolveWithFullResponse: true,
gzip: true, //使用gzip压缩让数据传输更快
headers: {
'User-Agent': 'Request-Promise',
}
};
rp(options).then((res)=> {
// 输出状态码
console.log(res.statusCode);
// 输出返回内容 (已自动进行gzip解压缩)
console.log(res.body)
}).catch((err) => {
// 错误处理
console.log("error")
});
request-promise(用户名密码认证)
使用提示
- http/https网页均可适用
- 请先安装
request-promise
和socks-proxy-agent
:npm install request-promise socks-proxy-agent
// 引入第三方 `socks-proxy-agent` 模块
const SocksProxyAgent = require('socks-proxy-agent');
// 引入第三方 `request-promise`
const rp = require('request-promise');
// 用户名密码认证(私密代理/独享代理)
const username = 'username';
const password = 'password';
// 代理服务器ip和端口
const proxy_ip = '59.38.241.25';
const proxy_port = 23918;
// 设置代理
let proxy = `socks5h://${username}:${password}@${proxy_ip}:${proxy_port}`
let agent = new SocksProxyAgent(proxy);
// 要访问的目标网页
let url = 'http://dev.kuaidaili.com/testproxy';
let options = {
uri: url,
agent: agent,
resolveWithFullResponse: true,
gzip: true, //使用gzip压缩让数据传输更快
headers: {
'User-Agent': 'Request-Promise'
}
};
rp(options).then((res)=> {
// 输出状态码
console.log(res.statusCode);
// 输出返回内容 (已自动进行gzip解压缩)
console.log(res.body)
}).catch((err) => {
console.log("error")
});
Puppeteer
Puppeteer(IP白名单)
使用提示
- http/https网页均可适用
- 运行环境要求: node7.6.0或以上 + puppeteer
- 请先安装puppeteer:
npm i puppeteer
puppeteer
模块的socks
代理暂不支持用户名密码的方式验证身份
// 引入puppeteer模块
const puppeteer = require('puppeteer');
// 要访问的目标网页
const url = 'http://dev.kuaidaili.com/testproxy';
// 添加headers
const headers = {
'Accept-Encoding': 'gzip'
};
// 代理服务器ip和端口
let proxy_ip = '59.38.241.25'
let proxy_port = 23918
(async ()=> {
// 新建一个浏览器实例
const browser = await puppeteer.launch({
headless: false, // 是否不显示窗口, 默认为true, 设为false便于调试
args: [
`--proxy-server=socks5h://${proxy_ip}:${proxy_port}`,
'--no-sandbox',
'--disable-setuid-sandbox'
]
});
// 打开一个新页面
const page = await browser.newPage();
// 设置headers
await page.setExtraHTTPHeaders(headers);
// 访问目标网页
await page.goto(url);
})();
Ruby
socksify
socksify(IP白名单)
require 'socksify/http' # 引入socksify/http模块
# 要访问的目标网页, 以快代理testproxy页面为例
page_url = "https://dev.kuaidaili.com/testproxy"
uri = URI(page_url)
# 代理服务器ip和端口
proxy_ip = '59.38.241.25'
proxy_port = 23918
# 新建socks代理实例
proxy = Net::HTTP.SOCKSProxy(proxy_ip, proxy_port)
# 创建新的请求对象
req = Net::HTTP::Get.new(uri)
# 设置User-Agent
req['User-Agent'] = 'Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50'
req['Accept-Encoding'] = 'gzip' # 使用gzip压缩传输数据让访问更快
# 使用代理发起请求, 若访问的是http网页, 请将use_ssl设为false
res = proxy.start(uri.hostname, uri.port, :use_ssl => true) do |http|
http.request(req)
end
# 输出状态码
puts "status code: #{res.code}"
# 输出响应体
if res.code.to_i != 200 then
puts "page content: #{res.body}"
else
gz = Zlib::GzipReader.new(StringIO.new(res.body.to_s))
puts "page content: #{gz.read}"
end
net/http
net/http(用户名密码认证)
# -*- coding: utf-8 -*-
require 'net/http' # 引入内置net/http模块
require 'zlib'
require 'stringio'
# 代理服务器ip和端口
proxy_ip = '59.38.241.25'
proxy_port = 23918
# 用户名密码认证(私密代理/独享代理)
username = 'username'
password = 'password'
# 要访问的目标网页, 以快代理testproxy页面为例
page_url = "https://dev.kuaidaili.com/testproxy"
uri = URI(page_url)
# 新建代理实例
proxy = Net::HTTP::Proxy(proxy_ip, proxy_port, username, password)
# 创建新的请求对象
req = Net::HTTP::Get.new(uri)
# 设置代理用户名密码认证
req.basic_auth(username, password)
# 设置User-Agent
req['User-Agent'] = 'Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50'
req['Accept-Encoding'] = 'gzip' # 使用gzip压缩传输数据让访问更快
# 使用代理发起请求, 若访问的是http网页, 请将use_ssl设为false
res = proxy.start(uri.hostname, uri.port, :use_ssl => true) do |http|
http.request(req)
end
# 输出状态码
puts "status code: #{res.code}"
# 输出响应体
if res.code.to_i != 200 then
puts "page content: #{res.body}"
else
gz = Zlib::GzipReader.new(StringIO.new(res.body.to_s))
puts "page content: #{gz.read}"
end
php
curl
curl
使用提示
- 此样例同时支持访问http和https网页
- curl不是php原生库,需要安装才能使用:
Ubuntu/Debian系统:apt-get install php5-curl
CentOS系统:yum install php-curl
<?php
// 要访问的目标页面
$targetUrl = "https://dev.kdlapi.com/testproxy";
// 代理ip
$proxyIp = "59.38.241.25";
$proxyPort = "23916";
$username = "username";
$password = "password";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $targetUrl);
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
// 设置代理
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5); //sock5
curl_setopt($ch, CURLOPT_PROXY, "socks5h://{$proxyIp}:{$proxyPort}");
//设置代理用户名密码
curl_setopt($ch, CURLOPT_PROXYAUTH, CURLAUTH_BASIC);
curl_setopt($ch, CURLOPT_PROXYUSERPWD, "{$username}:{$password}");
// 设置UA
curl_setopt($ch, CURLOPT_USERAGENT, "User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0);");
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
curl_setopt($ch, CURLOPT_TIMEOUT, 10);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$result = curl_exec($ch);
curl_close($ch);
var_dump($result);