3 Commits

Author SHA1 Message Date
wmq 00b0ebb775 fix(whatsapp): preserve history across reauthorization 2026-08-01 23:13:28 +08:00
wmq 785a5b196c fix(whatsapp): complete history synchronization 2026-08-01 20:23:17 +08:00
wmq cf25d71fcf feat(gateway): add scoped proxy egress deployment 2026-08-01 19:53:29 +08:00
20 changed files with 595 additions and 34 deletions
+5
View File
@@ -2,6 +2,9 @@
LX_DB_PASSWORD=replace-with-strong-database-password
LX_JWT_SECRET=replace-with-at-least-32-random-characters
LX_WEB_PORT=8088
DEBIAN_MIRROR=https://deb.debian.org/debian
DEBIAN_SECURITY_MIRROR=https://deb.debian.org/debian-security
FFMPEG_BINARIES_URL=https://github.com/eugeneware/ffmpeg-static/releases/download
# DeepSeek Anthropic-compatible API
DEEPSEEK_API_KEY=replace-with-rotated-deepseek-key
@@ -20,6 +23,8 @@ TELEGRAM_WEBHOOK_SECRET=replace-with-random-webhook-secret
WHATSAPP_GATEWAY_TOKEN=replace-with-at-least-32-random-characters
WHATSAPP_WEBHOOK_SECRET=replace-with-another-32-random-secret
WHATSAPP_LOG_LEVEL=info
# 可选。只影响 WhatsApp 网关出站,不设置系统或平台全局代理。
WHATSAPP_DEFAULT_PROXY_URL=
PUBLIC_WEBHOOK_BASE_URL=http://host.docker.internal:8088
APP_PUBLIC_URL=http://127.0.0.1:8088
+1
View File
@@ -28,6 +28,7 @@ storage/
*.sqlite3
*.dump
*.sql
!telegram-gateway/docker/scripts/init-db.sql
*.rdb
*.binlog
+20
View File
@@ -2,6 +2,26 @@
本项目遵循语义化版本。早期版本由经过脱敏的本地源码快照恢复,因此使用带日期的 `snapshot-*` 标签;自 `v1.0.0` 起按正式发布流程维护。
## [Unreleased]
- 修复 WhatsApp 账号重新授权后因网关账号 ID 与联系人名称变化导致历史会话分叉的问题;客户按稳定 JID 复用,旧渠道消息、任务、风险和商机自动归并,并保留最近有效的联系人资料。
### 新增
- Telegram Gateway 支持环境级默认 SOCKS5/HTTP/MTProto 代理,账号专属绑定仍具有更高优先级。
- WhatsApp Gateway 支持环境级默认 HTTP/HTTPS 代理,账号专属代理仍可覆盖默认值。
- 默认代理仅作用于两个消息网关,不改变平台、邮箱、数据库或 Docker 的全局网络出口。
- Docker 构建支持可配置的 Debian 与 FFmpeg 下载镜像,适配受限网络环境。
### 修复
- 将 Telegram Gateway 首次启动所需的 PostgreSQL 初始化脚本纳入版本控制,避免空数据库因挂载目录而启动失败。
- Telegram Gateway 本机调试端口改为 `127.0.0.1:3010`,避免与 Gitea 的 `3000` 端口冲突并阻止公网直连管理接口。
- 修复 WhatsApp `FULL` 历史通知被 Baileys 默认过滤的问题;首次连接和历史阶段完成后自动执行逐会话分页补齐。
- WhatsApp 历史同步增加并发合并、进度持久化、失败状态和平台数据库游标分页,避免重复同步及单次 2000 条截断。
- 修复 WhatsApp 先于平台 API 连通时首次同步被静默丢弃的问题,增加延迟启动、指数重试和重复 Webhook 恢复。
- WhatsApp 媒体与历史文件下载接入 Undici 代理 Dispatcher,并可恢复处理旧版本曾忽略但仍保存在缓存中的历史通知。
## [1.1.0] - 2026-08-01
### 新增
+1
View File
@@ -1,4 +1,5 @@
FROM node:20-alpine AS web-build
ARG FFMPEG_BINARIES_URL=https://github.com/eugeneware/ffmpeg-static/releases/download
WORKDIR /app
COPY package*.json ./
RUN npm ci
+6
View File
@@ -19,10 +19,14 @@ services:
whatsapp-gateway:
build:
context: ./whatsapp-gateway
args:
DEBIAN_MIRROR: ${DEBIAN_MIRROR:-https://deb.debian.org/debian}
DEBIAN_SECURITY_MIRROR: ${DEBIAN_SECURITY_MIRROR:-https://deb.debian.org/debian-security}
environment:
PORT: 3200
DATA_DIR: /data
WHATSAPP_GATEWAY_TOKEN: ${WHATSAPP_GATEWAY_TOKEN:?WHATSAPP_GATEWAY_TOKEN is required}
WHATSAPP_DEFAULT_PROXY_URL: ${WHATSAPP_DEFAULT_PROXY_URL:-}
LOG_LEVEL: ${WHATSAPP_LOG_LEVEL:-info}
volumes:
- lingxi_whatsapp_data:/data
@@ -89,6 +93,8 @@ services:
build:
context: .
dockerfile: Dockerfile
args:
FFMPEG_BINARIES_URL: ${FFMPEG_BINARIES_URL:-https://github.com/eugeneware/ffmpeg-static/releases/download}
ports:
- "${LX_WEB_PORT:-8088}:80"
depends_on:
+20
View File
@@ -37,6 +37,24 @@ docker compose logs --tail=100 api whatsapp-gateway web
平台的 `TELEGRAM_GATEWAY_URL` 指向网关 `/api` 地址。生产环境应通过私有网络连接,不把 Gateway 管理接口直接暴露公网。详细配置见 [Telegram Gateway 接入指南](TELEGRAM-GATEWAY.md)。
### 2.1 仅消息网关使用出站代理
生产环境可以创建一个不映射宿主机端口的专用 Docker 网络和 V2Ray/Xray 容器,并只将 Telegram、WhatsApp 网关接入该网络。不要设置 Docker Daemon、宿主机或平台 API 的全局 `HTTP_PROXY`/`HTTPS_PROXY`。
```env
# Telegram Gateway
TG_DEFAULT_PROXY_TYPE=socks5
TG_DEFAULT_PROXY_HOST=lingxi-egress
TG_DEFAULT_PROXY_PORT=10808
# 平台 WhatsApp Gateway
WHATSAPP_DEFAULT_PROXY_URL=http://lingxi-egress:10809
```
账号级代理优先于上述默认代理。代理容器应使用 `internal/external Docker network + no host ports` 的方式隔离;配置文件和认证信息仅保存在服务器受限目录,不进入 Git。上线前至少验证 Telegram API 与 WhatsApp Web 的 HTTPS 连接,并确认 API、数据库、邮箱和 AI 服务未配置全局代理变量。
若部署网络无法访问 Debian 官方源或 FFmpeg GitHub Release,可通过 `.env` 的 `DEBIAN_MIRROR`、`DEBIAN_SECURITY_MIRROR` 和 `FFMPEG_BINARIES_URL` 指定构建镜像;这些变量只影响镜像构建,不改变运行时出口。
## 3. 生产入口
在 Compose 外层部署 Caddy、Nginx 或云负载均衡:
@@ -49,6 +67,8 @@ docker compose logs --tail=100 api whatsapp-gateway web
将 `APP_PUBLIC_URL` 和 `PUBLIC_WEBHOOK_BASE_URL` 改为正式 HTTPS 地址,随后重建 API。
当前生产拓扑使用 Caddy 与 NPS 时,应在 NPS HTTP Host 中将域名指向平台内网 `8088`,Caddy 只反向代理到 NPS 的本机 HTTP 入口。修改前分别备份 NPS Host 配置和 Caddyfile,验证配置后再重启 NPS、平滑重载 Caddy。
## 4. 更新
```bash
+1
View File
@@ -18,6 +18,7 @@ Copy-Item telegram-gateway/.env.example telegram-gateway/.env
- `JWT_SECRET`、`ADMIN_API_KEY`:分别生成强随机值,不得复用。
- `DB_PASSWORD`:生产数据库强密码。
- `CORS_ORIGINS`:仅填写受信任的管理入口;平台服务端调用不依赖浏览器跨域。
- `TG_DEFAULT_PROXY_*`:可选的默认 TDLib 代理;账号绑定的专属代理优先。
生产环境不要使用示例中的 `postgres` 密码。`.env` 不得提交 Git。
+1 -1
View File
@@ -5,7 +5,7 @@
"type": "module",
"scripts": {
"start": "node src/index.js",
"test": "node --test test/business.test.js",
"test": "node --test test/business.test.js test/whatsapp-identity.test.js",
"test:email-protocol": "node --test test/email-protocol.test.js",
"test:email-api": "node --test test/email-api-e2e.test.js"
},
+283 -24
View File
@@ -18,6 +18,12 @@ import { conversationContext, deepseek } from './ai.js'
import { countBy, dateKey, percent, telegramAttachment } from './business.js'
import { resolveCustomerCountry } from './country.js'
import { knowledgeMetadata, rankKnowledge } from './knowledge.js'
import {
isWeakWhatsAppDisplayName,
reusableWhatsAppConversations,
whatsappConversationActivityAt,
whatsappCustomerStableId,
} from './whatsapp-identity.js'
import {
decryptEmailPassword,
encryptEmailPassword,
@@ -56,6 +62,8 @@ function broadcast(tenantId, event) {
const automaticTranslationQueues = { high: [], low: [] }
const automaticTranslationKeys = new Set()
const automaticTranslationCache = new Map()
const whatsappSyncStates = new Map()
const whatsappSyncSchedules = new Map()
let activeAutomaticTranslations = 0
let activeLowPriorityTranslations = 0
let lowPriorityResumeAt = 0
@@ -1473,18 +1481,27 @@ app.post('/api/conversations/whatsapp/open', async (req, res) => {
const identity = await whatsappRequest(
`/accounts/${channel.gatewayAccountId}/chats/${encodeURIComponent(jid)}/identity`,
)
const customerId = `wa-${channel.gatewayAccountId}-${jid}`
const conversationId = `${channel.id}:${jid}`
const existingCustomer = await resolveWhatsAppCustomer(
req.auth.tenantId,
channel,
jid,
conversationId,
)
const customerId = existingCustomer?.id || whatsappCustomerStableId(jid)
const customer = await putResource(req.auth.tenantId, 'customers', withDefaults('customers', {
...(existingCustomer || {}),
id: customerId,
name: identity.name || jid.split('@')[0],
phone: identity.phone || '',
avatarUrl: identity.avatarUrl || undefined,
name: !isWeakWhatsAppDisplayName(existingCustomer?.name, jid)
? existingCustomer.name
: identity.name || jid.split('@')[0],
phone: identity.phone || existingCustomer?.phone || '',
avatarUrl: identity.avatarUrl || existingCustomer?.avatarUrl || undefined,
whatsapp: jid,
whatsappChatType: identity.chatType || 'private',
channel: 'whatsapp',
source: 'whatsapp',
}))
const conversationId = `${channel.id}:${jid}`
await deleteResource(req.auth.tenantId, 'conversation-tombstones', conversationId)
const previous = await getResource(req.auth.tenantId, 'conversations', conversationId)
const conversation = await putResource(req.auth.tenantId, 'conversations', {
@@ -3144,6 +3161,32 @@ function whatsappMessageType(type) {
return 'text'
}
async function resolveWhatsAppCustomer(tenantId, channel, jid, conversationId) {
const current = await getResource(tenantId, 'conversations', conversationId)
if (current?.customerId) {
const customer = await getResource(tenantId, 'customers', current.customerId)
if (customer) return customer
}
const [channels, conversations] = await Promise.all([
listResources(tenantId, 'channels'),
listResources(tenantId, 'conversations'),
])
const activeChannelIds = new Set(channels.map((item) => String(item.id)))
const reusable = reusableWhatsAppConversations(
conversations,
activeChannelIds,
channel.id,
jid,
)
for (const conversation of reusable) {
if (!conversation.customerId) continue
const customer = await getResource(tenantId, 'customers', conversation.customerId)
if (customer) return customer
}
return getResource(tenantId, 'customers', whatsappCustomerStableId(jid))
}
async function upsertWhatsAppMessage(tenantId, channel, rawMessage, options = {}) {
const jid = String(rawMessage.chatId || '')
if (!jid || !rawMessage.id) return null
@@ -3170,15 +3213,19 @@ async function upsertWhatsAppMessage(tenantId, channel, rawMessage, options = {}
})
return { message, reaction: rawMessage, isNew: false }
}
const customerId = `wa-${channel.gatewayAccountId}-${jid}`
const existingCustomer = await getResource(tenantId, 'customers', customerId)
const existingCustomer = await resolveWhatsAppCustomer(tenantId, channel, jid, conversationId)
const customerId = existingCustomer?.id || whatsappCustomerStableId(jid)
const identity = options.identity || await whatsappRequest(
`/accounts/${channel.gatewayAccountId}/chats/${encodeURIComponent(jid)}/identity`,
).catch(() => null)
const customer = await putResource(tenantId, 'customers', withDefaults('customers', {
...(existingCustomer || {}),
id: customerId,
name: String(identity?.name || rawMessage.pushName || existingCustomer?.name || jid.split('@')[0]),
name: String(
!isWeakWhatsAppDisplayName(existingCustomer?.name, jid)
? existingCustomer.name
: identity?.name || rawMessage.pushName || existingCustomer?.name || jid.split('@')[0],
),
phone: String(identity?.phone || existingCustomer?.phone || ''),
avatarUrl: identity?.avatarUrl || existingCustomer?.avatarUrl,
whatsapp: jid,
@@ -3265,14 +3312,103 @@ async function upsertWhatsAppMessage(tenantId, channel, rawMessage, options = {}
return { message, customer, conversation, isNew: !existingMessage }
}
async function syncWhatsAppTenant(tenantId, channel, options = {}) {
async function mergeLegacyWhatsAppConversations(tenantId, channel, jid) {
const targetId = `${channel.id}:${jid}`
const target = await getResource(tenantId, 'conversations', targetId)
if (!target) return 0
const [channels, conversations] = await Promise.all([
listResources(tenantId, 'channels'),
listResources(tenantId, 'conversations'),
])
const activeChannelIds = new Set(channels.map((item) => String(item.id)))
const legacy = conversations.filter((item) => (
item.channel === 'whatsapp'
&& String(item.whatsappJid || '') === jid
&& String(item.id) !== targetId
&& !activeChannelIds.has(String(item.channelId || String(item.id).split(':')[0]))
))
if (!legacy.length) return 0
let imported = 0
let earliestCreatedAt = target.createdAt
let lastMessage = target.lastMessage
let customerId = target.customerId
let canonicalActivityAt = whatsappConversationActivityAt(target)
const mergedCustomerIds = new Set(target.customerId ? [target.customerId] : [])
const mergedIds = new Set(target.mergedLegacyConversationIds || [])
for (const source of legacy) {
if (source.customerId) mergedCustomerIds.add(source.customerId)
const sourceActivityAt = whatsappConversationActivityAt(source)
if (source.customerId && sourceActivityAt >= canonicalActivityAt) {
customerId = source.customerId
canonicalActivityAt = sourceActivityAt
}
const messages = await listResources(tenantId, `messages:${source.id}`)
for (const message of messages) {
const exists = await getResource(tenantId, `messages:${targetId}`, message.id)
await putResource(tenantId, `messages:${targetId}`, {
...message,
conversationId: targetId,
})
if (!exists) imported++
if (!lastMessage || Date.parse(message.createdAt || '') > Date.parse(lastMessage.createdAt || '')) {
lastMessage = { ...message, conversationId: targetId }
}
}
if (!earliestCreatedAt || Date.parse(source.createdAt || '') < Date.parse(earliestCreatedAt)) {
earliestCreatedAt = source.createdAt
}
mergedIds.add(source.id)
for (const message of messages) {
await deleteResource(tenantId, `messages:${source.id}`, message.id)
}
await deleteResource(tenantId, 'conversations', source.id)
}
await patchResource(tenantId, 'conversations', targetId, {
customerId,
createdAt: earliestCreatedAt || target.createdAt,
lastMessage,
mergedLegacyConversationIds: [...mergedIds],
historySyncedAt: new Date().toISOString(),
})
for (const duplicateCustomerId of mergedCustomerIds) {
if (!duplicateCustomerId || duplicateCustomerId === customerId) continue
for (const kind of ['opportunities', 'tasks', 'risks']) {
const resources = await listResources(tenantId, kind)
for (const resource of resources.filter((item) => item.customerId === duplicateCustomerId)) {
await patchResource(tenantId, kind, resource.id, { customerId })
}
}
const remainingConversations = (await listResources(tenantId, 'conversations'))
.some((item) => item.customerId === duplicateCustomerId)
const remainingBusinessReferences = (await Promise.all(
['opportunities', 'tasks', 'risks'].map((kind) => listResources(tenantId, kind)),
)).some((items) => items.some((item) => item.customerId === duplicateCustomerId))
if (!remainingConversations && !remainingBusinessReferences) {
await deleteResource(tenantId, 'customers', duplicateCustomerId)
}
}
return imported
}
async function performWhatsAppTenantSync(tenantId, channel, options = {}) {
const state = await whatsappRequest(`/accounts/${channel.gatewayAccountId}/auth/state`)
if (state.status !== 'connected' || !['open', 'ready'].includes(state.connectionState)) {
throw fail(409, 'WhatsApp 账号尚未完成关联设备配对')
}
const chats = await whatsappRequest(`/accounts/${channel.gatewayAccountId}/chats`)
let messagesSynced = 0
for (const chat of chats || []) {
if (options.full === true) {
await patchResource(tenantId, 'channels', channel.id, {
historySyncStatus: 'running',
historySyncStartedAt: new Date().toISOString(),
historySyncChatsCompleted: 0,
historySyncChatsTotal: chats?.length || 0,
historySyncMessagesSynced: 0,
historySyncError: undefined,
})
}
for (const [chatIndex, chat] of (chats || []).entries()) {
const jid = String(chat.id || '')
if (!jid) continue
if (options.full === true) {
@@ -3280,20 +3416,29 @@ async function syncWhatsAppTenant(tenantId, channel, options = {}) {
`/accounts/${channel.gatewayAccountId}/chats/${encodeURIComponent(jid)}/history/sync`,
{
method: 'POST',
body: JSON.stringify({ pages: 20, count: 50 }),
body: JSON.stringify({ pages: 50, count: 50 }),
},
).catch(() => null)
}
const identity = await whatsappRequest(
`/accounts/${channel.gatewayAccountId}/chats/${encodeURIComponent(jid)}/identity`,
).catch(() => ({ name: chat.name || jid.split('@')[0] }))
const history = await whatsappRequest(
`/accounts/${channel.gatewayAccountId}/chats/${encodeURIComponent(jid)}/history?limit=2000`,
).catch(() => ({ messages: [] }))
for (const raw of history.messages || []) {
const result = await upsertWhatsAppMessage(tenantId, channel, raw, { identity, notify: false })
if (result?.isNew) messagesSynced++
}
let historyCursor = ''
let historyPages = 0
do {
const history = await whatsappRequest(
`/accounts/${channel.gatewayAccountId}/chats/${encodeURIComponent(jid)}/history?limit=2000${historyCursor ? `&before=${encodeURIComponent(historyCursor)}` : ''}`,
).catch(() => ({ messages: [], beforeCursor: null, hasMoreBefore: false }))
for (const raw of history.messages || []) {
const result = await upsertWhatsAppMessage(tenantId, channel, raw, { identity, notify: false })
if (result?.isNew) messagesSynced++
}
historyPages++
historyCursor = options.full === true && history.hasMoreBefore && history.beforeCursor
? String(history.beforeCursor)
: ''
} while (historyCursor && historyPages < 100)
messagesSynced += await mergeLegacyWhatsAppConversations(tenantId, channel, jid)
const conversationId = `${channel.id}:${jid}`
const current = await getResource(tenantId, 'conversations', conversationId)
if (current) {
@@ -3302,12 +3447,27 @@ async function syncWhatsAppTenant(tenantId, channel, options = {}) {
historySyncedAt: new Date().toISOString(),
})
}
if (options.full === true) {
await patchResource(tenantId, 'channels', channel.id, {
historySyncChatsCompleted: chatIndex + 1,
historySyncChatsTotal: chats.length,
historySyncMessagesSynced: messagesSynced,
historySyncUpdatedAt: new Date().toISOString(),
})
}
}
await patchResource(tenantId, 'channels', channel.id, {
lastSyncAt: new Date().toISOString(),
status: 'connected',
authState: 'connected',
errorMessage: undefined,
...(options.full === true ? {
historySyncStatus: 'completed',
historySyncCompletedAt: new Date().toISOString(),
historySyncChatsCompleted: chats?.length || 0,
historySyncChatsTotal: chats?.length || 0,
historySyncMessagesSynced: messagesSynced,
} : {}),
})
await appendChannelLog(
tenantId,
@@ -3323,6 +3483,76 @@ async function syncWhatsAppTenant(tenantId, channel, options = {}) {
return { chats: chats?.length || 0, messages: messagesSynced }
}
async function syncWhatsAppTenant(tenantId, channel, options = {}) {
const key = `${tenantId}:${channel.id}`
const existing = whatsappSyncStates.get(key)
if (existing) {
if (options.full === true && !existing.runningFull) existing.fullRequested = true
return existing.promise
}
const syncState = {
fullRequested: options.full === true,
runningFull: false,
promise: null,
}
syncState.promise = (async () => {
let result = { chats: 0, messages: 0 }
try {
do {
const full = syncState.fullRequested
syncState.fullRequested = false
syncState.runningFull = full
result = await performWhatsAppTenantSync(tenantId, channel, { full })
syncState.runningFull = false
} while (syncState.fullRequested)
return result
} catch (error) {
if (syncState.runningFull) {
await patchResource(tenantId, 'channels', channel.id, {
historySyncStatus: 'failed',
historySyncError: String(error?.message || error),
historySyncUpdatedAt: new Date().toISOString(),
}).catch(() => null)
}
throw error
} finally {
whatsappSyncStates.delete(key)
}
})()
whatsappSyncStates.set(key, syncState)
return syncState.promise
}
function scheduleWhatsAppSync(tenantId, channel, options = {}, attempt = 0) {
const key = `${tenantId}:${channel.id}`
const scheduled = whatsappSyncSchedules.get(key)
if (scheduled) {
if (options.full === true) scheduled.full = true
return
}
const state = { full: options.full === true, timer: null }
const delay = attempt === 0 ? 2_000 : Math.min(60_000, 2_000 * (2 ** attempt))
state.timer = setTimeout(async () => {
whatsappSyncSchedules.delete(key)
try {
await syncWhatsAppTenant(tenantId, channel, { full: state.full })
} catch (error) {
if (attempt < 6) {
scheduleWhatsAppSync(tenantId, channel, { full: state.full }, attempt + 1)
} else {
await patchResource(tenantId, 'channels', channel.id, {
historySyncStatus: 'failed',
historySyncError: String(error?.message || error),
historySyncUpdatedAt: new Date().toISOString(),
}).catch(() => null)
}
}
}, delay)
state.timer.unref?.()
whatsappSyncSchedules.set(key, state)
}
async function syncTelegramTenant(tenantId, channel) {
const gateway = await gatewayRequest(`/tg-accounts/${channel.gatewayAccountId}/chats/sync`)
const chats = await gatewayRequest(`/tg-accounts/${channel.gatewayAccountId}/chats?limit=100`)
@@ -4530,6 +4760,14 @@ app.post('/internal/webhooks/whatsapp/:tenantId/:channelId', async (req, res) =>
|| crypto.createHash('sha256').update(req.rawBody || Buffer.from(JSON.stringify(req.body))).digest('hex'),
)
if (await getResource(tenantId, 'webhook-events', eventId)) {
if (req.body.event === 'account.connected') {
scheduleWhatsAppSync(tenantId, channel, { full: true })
} else if (req.body.event === 'history.synced') {
const historyReady = req.body?.data?.isLatest === true || Number(req.body?.data?.progress) === 100
scheduleWhatsAppSync(tenantId, channel, { full: historyReady })
} else if (req.body.event === 'history.status' && req.body?.data?.status === 'complete') {
scheduleWhatsAppSync(tenantId, channel, { full: true })
}
return ok(res, { accepted: true, duplicate: true })
}
await putResource(tenantId, 'webhook-events', {
@@ -4644,14 +4882,17 @@ app.post('/internal/webhooks/whatsapp/:tenantId/:channelId', async (req, res) =>
for (const contact of req.body?.data?.contacts || []) {
const jid = String(contact.id || '')
if (!jid) continue
const customerId = `wa-${channel.gatewayAccountId}-${jid}`
const existing = await getResource(tenantId, 'customers', customerId)
const conversationId = `${channel.id}:${jid}`
const existing = await resolveWhatsAppCustomer(tenantId, channel, jid, conversationId)
if (!existing) continue
const customerId = existing.id
const identity = await whatsappRequest(
`/accounts/${channel.gatewayAccountId}/chats/${encodeURIComponent(jid)}/identity`,
).catch(() => null)
const customer = await patchResource(tenantId, 'customers', customerId, {
name: identity?.name || contact.name || contact.notify || contact.verifiedName || existing.name,
name: !isWeakWhatsAppDisplayName(existing.name, jid)
? existing.name
: identity?.name || contact.name || contact.notify || contact.verifiedName || existing.name,
phone: identity?.phone || existing.phone,
avatarUrl: identity?.avatarUrl || existing.avatarUrl,
whatsappVerifiedName: contact.verifiedName || existing.whatsappVerifiedName,
@@ -4683,9 +4924,25 @@ app.post('/internal/webhooks/whatsapp/:tenantId/:channelId', async (req, res) =>
createdAt: new Date().toISOString(),
})
}
if (connected) void syncWhatsAppTenant(tenantId, channel).catch(() => null)
if (connected) scheduleWhatsAppSync(tenantId, channel, { full: true })
} else if (req.body.event === 'history.synced') {
void syncWhatsAppTenant(tenantId, channel).catch(() => null)
const historyReady = req.body?.data?.isLatest === true || Number(req.body?.data?.progress) === 100
scheduleWhatsAppSync(tenantId, channel, { full: historyReady })
} else if (req.body.event === 'history.status') {
await patchResource(tenantId, 'channels', channelId, {
gatewayHistoryStatus: req.body?.data?.status,
gatewayHistoryProgress: req.body?.data?.progress,
gatewayHistorySyncType: req.body?.data?.syncType,
gatewayHistoryUpdatedAt: req.body?.data?.updatedAt || new Date().toISOString(),
})
broadcast(tenantId, {
type: 'channel.history.updated',
channelId,
history: req.body.data,
})
if (req.body?.data?.status === 'complete') {
scheduleWhatsAppSync(tenantId, channel, { full: true })
}
}
ok(res, { accepted: true })
})
@@ -4779,7 +5036,9 @@ for (const row of existingWhatsAppChannels.rows) {
const tenantId = String(row.tenant_id)
await ensureWhatsAppWebhook(tenantId, row.data, row.data.status !== 'paused').then(async (updatedChannel) => {
if (row.data.status !== 'paused') {
void syncWhatsAppTenant(tenantId, updatedChannel).catch(() => null)
scheduleWhatsAppSync(tenantId, updatedChannel, {
full: row.data.historySyncStatus !== 'completed',
})
}
}).catch(async (error) => {
await patchResource(tenantId, 'channels', row.data.id, {
+36
View File
@@ -0,0 +1,36 @@
import crypto from 'node:crypto'
export function whatsappCustomerStableId(jid) {
const normalized = String(jid || '').trim().toLowerCase()
const hash = crypto.createHash('sha256').update(normalized).digest('hex').slice(0, 24)
return `wa-customer-${hash}`
}
export function whatsappConversationActivityAt(conversation) {
return Date.parse(
conversation?.lastMessage?.createdAt
|| conversation?.updatedAt
|| conversation?.createdAt
|| '',
) || 0
}
export function isWeakWhatsAppDisplayName(value, jid) {
const name = String(value || '').trim()
const jidName = String(jid || '').split('@')[0].replace(/\D/g, '')
const digits = name.replace(/\D/g, '')
return !name || (digits.length >= 7 && (!jidName || digits === jidName))
}
export function reusableWhatsAppConversations(conversations, activeChannelIds, channelId, jid) {
return conversations
.filter((item) => (
item.channel === 'whatsapp'
&& String(item.whatsappJid || '') === String(jid)
&& (
String(item.channelId || '') === String(channelId)
|| !activeChannelIds.has(String(item.channelId || String(item.id).split(':')[0]))
)
))
.sort((left, right) => whatsappConversationActivityAt(right) - whatsappConversationActivityAt(left))
}
+32
View File
@@ -0,0 +1,32 @@
import assert from 'node:assert/strict'
import test from 'node:test'
import {
isWeakWhatsAppDisplayName,
reusableWhatsAppConversations,
whatsappCustomerStableId,
} from '../src/whatsapp-identity.js'
test('WhatsApp customer id remains stable after gateway reauthorization', () => {
assert.equal(
whatsappCustomerStableId('15551234567@s.whatsapp.net'),
whatsappCustomerStableId('15551234567@s.whatsapp.net'),
)
assert.match(whatsappCustomerStableId('15551234567@s.whatsapp.net'), /^wa-customer-[a-f0-9]{24}$/)
})
test('the latest inactive conversation is reused while another active account stays isolated', () => {
const jid = '15551234567@s.whatsapp.net'
const conversations = [
{ id: 'old:newest', channel: 'whatsapp', channelId: 'old', whatsappJid: jid, customerId: 'abigail', lastMessage: { createdAt: '2026-07-31T07:51:37.000Z' } },
{ id: 'older:history', channel: 'whatsapp', channelId: 'older', whatsappJid: jid, customerId: 'legacy', lastMessage: { createdAt: '2026-07-29T17:25:42.000Z' } },
{ id: 'other:active', channel: 'whatsapp', channelId: 'other', whatsappJid: jid, customerId: 'other-account', lastMessage: { createdAt: '2026-08-01T00:00:00.000Z' } },
]
const result = reusableWhatsAppConversations(conversations, new Set(['current', 'other']), 'current', jid)
assert.deepEqual(result.map((item) => item.customerId), ['abigail', 'legacy'])
})
test('a saved contact name is not replaced by a phone-number fallback', () => {
const jid = '15551234567@s.whatsapp.net'
assert.equal(isWeakWhatsAppDisplayName('Abigail', jid), false)
assert.equal(isWeakWhatsAppDisplayName('+1 555 123 4567', jid), true)
})
+11
View File
@@ -1,5 +1,8 @@
# Application
NODE_ENV=development
# Optional image build mirrors for networks where deb.debian.org is unavailable.
DEBIAN_MIRROR=https://deb.debian.org/debian
DEBIAN_SECURITY_MIRROR=https://deb.debian.org/debian-security
PORT=3000
APP_NAME=TelegramUserGateway
CORS_ORIGINS=
@@ -29,6 +32,14 @@ TDLIB_PATH=/usr/local/lib/libtdjson.so
TG_API_ID=0
TG_API_HASH=your_api_hash_here
# 可选默认代理。账号未绑定专属代理时使用;只影响 Telegram TDLib 出站。
TG_DEFAULT_PROXY_TYPE=
TG_DEFAULT_PROXY_HOST=
TG_DEFAULT_PROXY_PORT=
TG_DEFAULT_PROXY_USERNAME=
TG_DEFAULT_PROXY_PASSWORD=
TG_DEFAULT_PROXY_SECRET=
# Rate Limiting
RATE_LIMIT_DEFAULT_TTL=60000
RATE_LIMIT_DEFAULT_LIMIT=30
+6
View File
@@ -1,5 +1,8 @@
FROM node:20-bookworm-slim AS builder
ARG DEBIAN_MIRROR=https://deb.debian.org/debian
ARG DEBIAN_SECURITY_MIRROR=https://deb.debian.org/debian-security
RUN sed -i "s|http://deb.debian.org/debian-security|${DEBIAN_SECURITY_MIRROR}|g; s|http://deb.debian.org/debian|${DEBIAN_MIRROR}|g" /etc/apt/sources.list.d/debian.sources
RUN apt-get update \
&& apt-get install -y --no-install-recommends g++ make python3 \
&& rm -rf /var/lib/apt/lists/*
@@ -22,6 +25,9 @@ RUN npm prune --omit=dev
FROM node:20-bookworm-slim
ARG DEBIAN_MIRROR=https://deb.debian.org/debian
ARG DEBIAN_SECURITY_MIRROR=https://deb.debian.org/debian-security
RUN sed -i "s|http://deb.debian.org/debian-security|${DEBIAN_SECURITY_MIRROR}|g; s|http://deb.debian.org/debian|${DEBIAN_MIRROR}|g" /etc/apt/sources.list.d/debian.sources
RUN apt-get update \
&& apt-get install -y --no-install-recommends tini ca-certificates \
&& rm -rf /var/lib/apt/lists/*
+10 -1
View File
@@ -30,9 +30,12 @@ services:
build:
context: ..
dockerfile: docker/Dockerfile
args:
DEBIAN_MIRROR: ${DEBIAN_MIRROR:-https://deb.debian.org/debian}
DEBIAN_SECURITY_MIRROR: ${DEBIAN_SECURITY_MIRROR:-https://deb.debian.org/debian-security}
container_name: tgw-app
ports:
- '3000:3000'
- '127.0.0.1:3010:3000'
env_file:
- ../.env
environment:
@@ -52,6 +55,12 @@ services:
TDLIB_PATH: /app/node_modules/@prebuilt-tdlib/linux-x64-glibc/libtdjson.so
TG_API_ID: ${TG_API_ID}
TG_API_HASH: ${TG_API_HASH}
TG_DEFAULT_PROXY_TYPE: ${TG_DEFAULT_PROXY_TYPE:-}
TG_DEFAULT_PROXY_HOST: ${TG_DEFAULT_PROXY_HOST:-}
TG_DEFAULT_PROXY_PORT: ${TG_DEFAULT_PROXY_PORT:-}
TG_DEFAULT_PROXY_USERNAME: ${TG_DEFAULT_PROXY_USERNAME:-}
TG_DEFAULT_PROXY_PASSWORD: ${TG_DEFAULT_PROXY_PASSWORD:-}
TG_DEFAULT_PROXY_SECRET: ${TG_DEFAULT_PROXY_SECRET:-}
depends_on:
postgres:
condition: service_healthy
@@ -0,0 +1,8 @@
-- Telegram User Gateway - Database Initialization
-- This script runs on first PostgreSQL startup
-- Enable UUID extension
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
-- Create indexes will be handled by TypeORM synchronize in dev
-- or by migrations in production
@@ -171,7 +171,7 @@ export class TgAccountsService implements OnModuleInit {
const binding = await this.bindingRepo.findOne({
where: { accountId, isActive: true },
});
if (!binding) return undefined;
if (!binding) return this.getDefaultProxy();
const proxy = await this.proxyRepo
.createQueryBuilder('proxy')
@@ -179,7 +179,7 @@ export class TgAccountsService implements OnModuleInit {
.where('proxy.id = :id', { id: binding.proxyId })
.andWhere('proxy.isActive = :isActive', { isActive: true })
.getOne();
if (!proxy) return undefined;
if (!proxy) return this.getDefaultProxy();
return {
type: proxy.type,
@@ -190,4 +190,27 @@ export class TgAccountsService implements OnModuleInit {
secret: proxy.secret,
};
}
private getDefaultProxy() {
const host = String(process.env.TG_DEFAULT_PROXY_HOST || '').trim();
const port = Number(process.env.TG_DEFAULT_PROXY_PORT || 0);
if (!host || !Number.isInteger(port) || port < 1 || port > 65535) {
return undefined;
}
const type = String(process.env.TG_DEFAULT_PROXY_TYPE || 'socks5').toLowerCase();
if (!['socks5', 'http', 'mtproto'].includes(type)) {
this.logger.warn(`Ignoring unsupported TG_DEFAULT_PROXY_TYPE: ${type}`);
return undefined;
}
return {
type,
host,
port,
username: process.env.TG_DEFAULT_PROXY_USERNAME || '',
password: process.env.TG_DEFAULT_PROXY_PASSWORD || '',
secret: process.env.TG_DEFAULT_PROXY_SECRET || '',
};
}
}
+4
View File
@@ -1,5 +1,9 @@
FROM node:20-bookworm-slim
ARG DEBIAN_MIRROR=https://deb.debian.org/debian
ARG DEBIAN_SECURITY_MIRROR=https://deb.debian.org/debian-security
RUN sed -i "s|http://deb.debian.org/debian-security|${DEBIAN_SECURITY_MIRROR}|g; s|http://deb.debian.org/debian|${DEBIAN_MIRROR}|g" /etc/apt/sources.list.d/debian.sources
ENV NODE_ENV=production
WORKDIR /app
+11 -1
View File
@@ -15,7 +15,8 @@
"https-proxy-agent": "^7.0.6",
"multer": "2.2.0",
"pino": "9.6.0",
"qrcode": "1.5.4"
"qrcode": "1.5.4",
"undici": "^6.21.3"
}
},
"node_modules/@borewit/text-codec": {
@@ -2624,6 +2625,15 @@
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/undici": {
"version": "6.21.3",
"resolved": "https://registry.npmmirror.com/undici/-/undici-6.21.3.tgz",
"integrity": "sha512-gBLkYIlEnSp8pFbT64yFgGE6UIB9tAkhukC23PmMDCe5Nd+cRqKxSjw5y54MK2AZMgZfJWMaNE4nYUHgi1XEOw==",
"license": "MIT",
"engines": {
"node": ">=18.17"
}
},
"node_modules/undici-types": {
"version": "8.3.0",
"resolved": "https://registry.npmmirror.com/undici-types/-/undici-types-8.3.0.tgz",
+2 -1
View File
@@ -17,6 +17,7 @@
"https-proxy-agent": "^7.0.6",
"multer": "2.2.0",
"pino": "9.6.0",
"qrcode": "1.5.4"
"qrcode": "1.5.4",
"undici": "^6.21.3"
}
}
+112 -4
View File
@@ -9,10 +9,12 @@ import path from 'node:path'
import https from 'node:https'
import { Boom } from '@hapi/boom'
import { HttpsProxyAgent } from 'https-proxy-agent'
import { ProxyAgent as UndiciProxyAgent } from 'undici'
import makeWASocket, {
Browsers,
BufferJSON,
DisconnectReason,
downloadAndProcessHistorySyncNotification,
downloadMediaMessage,
fetchLatestBaileysVersion,
getContentType,
@@ -31,6 +33,7 @@ const accountsFile = path.join(dataDir, 'accounts.json')
const webhookDeliveriesFile = path.join(dataDir, 'webhook-deliveries.json')
const outboundJobsFile = path.join(dataDir, 'outbound-jobs.json')
const gatewayToken = String(process.env.WHATSAPP_GATEWAY_TOKEN || '')
const defaultProxyUrl = String(process.env.WHATSAPP_DEFAULT_PROXY_URL || '').trim()
const maxMessagesPerChat = Math.max(2_000, Number(process.env.MAX_MESSAGES_PER_CHAT || 20_000))
const log = pino({ level: process.env.LOG_LEVEL || 'info' })
@@ -90,7 +93,7 @@ function accountWebhookSecret(account) {
}
function accountProxyUrl(account) {
return decryptCredential(account?.proxyUrlEncrypted || '')
return decryptCredential(account?.proxyUrlEncrypted || '') || defaultProxyUrl
}
function jsonSafe(value) {
@@ -307,6 +310,61 @@ function upsertMessage(accountId, raw) {
return message
}
async function replayStoredHistoryNotifications(
accountId, account, store, logger, processedIds = new Set(), downloadOptions = {},
) {
const replayed = new Set(account.replayedHistoryNotificationIds || [])
const candidates = []
for (const messages of store.messages.values()) {
for (const item of messages || []) {
const notification = item.raw?.message?.protocolMessage?.historySyncNotification
if (!notification || !item.id || replayed.has(item.id) || processedIds.has(item.id)) continue
candidates.push({ id: item.id, notification })
}
}
for (const candidate of candidates) {
try {
const data = await downloadAndProcessHistorySyncNotification(
candidate.notification, downloadOptions, logger,
)
for (const chat of data.chats || []) {
const id = String(chat.id || '')
if (id) store.chats.set(id, { ...(store.chats.get(id) || {}), ...jsonSafe(chat), id })
}
for (const contact of data.contacts || []) {
const id = String(contact.id || '')
if (id) store.contacts.set(id, { ...(store.contacts.get(id) || {}), ...jsonSafe(contact), id })
}
for (const message of data.messages || []) upsertMessage(accountId, message)
replayed.add(candidate.id)
account.replayedHistoryNotificationIds = [...replayed].slice(-100)
account.historySync = {
status: 'processing',
syncType: data.syncType ?? candidate.notification.syncType ?? null,
progress: data.progress ?? candidate.notification.progress ?? null,
chats: data.chats?.length || 0,
contacts: data.contacts?.length || 0,
messages: data.messages?.length || 0,
recovered: true,
updatedAt: new Date().toISOString(),
}
await saveAccounts()
scheduleStoreSave(accountId)
void deliverWebhook(account, 'history.synced', {
chats: data.chats?.length || 0,
contacts: data.contacts?.length || 0,
messages: data.messages?.length || 0,
progress: data.progress ?? candidate.notification.progress ?? null,
syncType: data.syncType ?? candidate.notification.syncType ?? null,
recovered: true,
isLatest: true,
})
} catch (error) {
logger.warn({ err: error, messageId: candidate.id }, 'stored history notification replay failed')
}
}
}
function storedMessage(accountId, jid, messageId) {
const store = stores.get(accountId)
return (store?.messages.get(jid) || []).find((item) => item.id === messageId)
@@ -671,10 +729,13 @@ async function connectAccount(accountId) {
const authPath = path.join(authDir, accountId)
await fs.mkdir(authPath, { recursive: true, mode: 0o700 })
const { state, saveCreds } = await useMultiFileAuthState(authPath)
const { version } = await fetchLatestBaileysVersion()
const logger = log.child({ accountId })
const proxyUrl = accountProxyUrl(account)
const proxyAgent = proxyUrl ? new HttpsProxyAgent(proxyUrl) : undefined
const fetchDispatcher = proxyUrl ? new UndiciProxyAgent(proxyUrl) : undefined
const { version } = await fetchLatestBaileysVersion(
fetchDispatcher ? { dispatcher: fetchDispatcher } : {},
)
const sock = makeWASocket({
version,
logger,
@@ -684,7 +745,11 @@ async function connectAccount(accountId) {
},
browser: Browsers.ubuntu('Lingxi Crossborder'),
...(proxyAgent ? { agent: proxyAgent, fetchAgent: proxyAgent } : {}),
...(fetchDispatcher ? { options: { dispatcher: fetchDispatcher } } : {}),
syncFullHistory: true,
// Baileys 7 excludes FULL history notifications by default even when
// syncFullHistory is enabled. Accept every server-provided history phase.
shouldSyncHistoryMessage: () => true,
markOnlineOnConnect: false,
generateHighQualityLinkPreview: true,
getMessage: async (key) => {
@@ -693,7 +758,7 @@ async function connectAccount(accountId) {
},
cachedGroupMetadata: async (jid) => store.groups.get(jid),
})
sessions.set(accountId, { sock, connecting: true })
sessions.set(accountId, { sock, connecting: true, fetchDispatcher })
account.status = state.creds.registered ? 'connecting' : 'waiting_pairing'
account.updatedAt = new Date().toISOString()
await saveAccounts()
@@ -721,6 +786,18 @@ async function connectAccount(accountId) {
const session = sessions.get(accountId)
if (session) session.connecting = false
await saveAccounts()
const processedIds = new Set(
(state.creds.processedHistoryMessages || []).map((item) => String(item?.key?.id || '')),
)
void replayStoredHistoryNotifications(
accountId,
account,
store,
logger,
processedIds,
fetchDispatcher ? { dispatcher: fetchDispatcher } : {},
)
.catch((error) => logger.warn({ err: error }, 'stored history recovery failed'))
for (const job of outboundJobs.values()) {
if (job.accountId === accountId && ['queued', 'retrying', 'sending'].includes(job.status)) {
scheduleOutboundJob(job.id)
@@ -732,7 +809,9 @@ async function connectAccount(accountId) {
if (sessions.get(accountId)?.sock !== sock) return
const statusCode = new Boom(update.lastDisconnect?.error).output?.statusCode
const loggedOut = statusCode === DisconnectReason.loggedOut
const session = sessions.get(accountId)
sessions.delete(accountId)
if (session?.fetchDispatcher) void session.fetchDispatcher.close().catch(() => undefined)
account.status = loggedOut ? 'logged_out' : 'disconnected'
account.connectionState = loggedOut ? 'logged_out' : 'reconnecting'
account.lastError = update.lastDisconnect?.error?.message || `连接关闭 (${statusCode || 'unknown'})`
@@ -752,7 +831,9 @@ async function connectAccount(accountId) {
}
}
})
sock.ev.on('messaging-history.set', ({ chats = [], contacts = [], messages = [] }) => {
sock.ev.on('messaging-history.set', ({
chats = [], contacts = [], messages = [], isLatest, progress, syncType, chunkOrder,
}) => {
for (const chat of chats) {
const id = String(chat.id || '')
if (id) store.chats.set(id, { ...(store.chats.get(id) || {}), ...jsonSafe(chat), id })
@@ -762,13 +843,40 @@ async function connectAccount(accountId) {
if (id) store.contacts.set(id, { ...(store.contacts.get(id) || {}), ...jsonSafe(contact), id })
}
for (const item of messages) upsertMessage(accountId, item)
account.historySync = {
status: isLatest || progress === 100 ? 'processing' : 'syncing',
syncType: syncType ?? null,
progress: progress ?? null,
chunkOrder: chunkOrder ?? null,
chats: chats.length,
contacts: contacts.length,
messages: messages.length,
updatedAt: new Date().toISOString(),
}
void saveAccounts()
scheduleStoreSave(accountId)
void deliverWebhook(account, 'history.synced', {
chats: chats.length,
contacts: contacts.length,
messages: messages.length,
isLatest: Boolean(isLatest),
progress: progress ?? null,
syncType: syncType ?? null,
chunkOrder: chunkOrder ?? null,
})
})
sock.ev.on('messaging-history.status', ({ syncType, status, explicit }) => {
account.historySync = {
...(account.historySync || {}),
status,
syncType,
explicit: Boolean(explicit),
updatedAt: new Date().toISOString(),
...(status === 'complete' ? { completedAt: new Date().toISOString() } : {}),
}
void saveAccounts()
void deliverWebhook(account, 'history.status', account.historySync)
})
sock.ev.on('chats.upsert', (items) => {
for (const chat of items || []) {
const id = String(chat.id || '')