Caileb/Checkpoint-Golang
Archived
1
0
Fork 0
Code Activity

Import existing project

Browse source
This commit is contained in:
Caileb 2025-05-26 12:42:36 -05:00
parent 7887817595
commit 80b0cc4939
125 changed files with 16980 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

743
develop/html/checkpoint.html Normal file
View file

@ -0,0 +1,743 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset=UTF-8>
<meta http-equiv=X-UA-Compatible content="IE=edge">
<meta name=viewport content="width=device-width,initial-scale=1">
<title>Checkpoint Documentation</title>
<meta name=description content="Documentation for the Checkpoint Protection System, a secure Proof-of-Work solution to prevent automated abuse.">
<link rel=icon href=/images/favi.png type=image/png>
<link rel=apple-touch-icon href=/images/favi.png>
<link rel="shortcut icon" href=/images/favi.png>
<link rel=preload href=/webfonts/Poppins-Regular.woff2 as=font type=font/woff2 crossorigin>
<link rel=preload href=/webfonts/Poppins-SemiBold.woff2 as=font type=font/woff2 crossorigin>
<link rel="preload" href="/js/u.js" as="script">
<link rel="stylesheet" href="/css/u.css">
<link rel="stylesheet" href="/css/docs.css">
<link rel="stylesheet" href="https://unpkg.com/@speed-highlight/core@1.2.7/dist/themes/github-dark.css">
<style>
:root {
--background-color: #1a1a1a;
--overlay-bg: rgba(28, 28, 28, 0.95);
--text-color: #fff;
--subtext-color: #ccc;
--accent-color: #9B59B6;
}
body {
background-color: var(--background-color);
color: var(--text-color);
font-family: 'Poppins', sans-serif;
}
.container {
background: rgba(30, 30, 30, 0.85);
backdrop-filter: blur(8px);
border-radius: 20px;
padding: 20px;
margin: 40px auto;
box-shadow: 0 10px 30px rgba(0,0,0,0.3), 0 1px 2px rgba(155,89,182,0.2);
max-width: 960px;
}
</style>
<script async src=/js/u.js></script>
<script type="module" src="/js/docs.js"></script>
<script async src="/js/lightbox.js"></script>
</head>
<body>
<div class="container">
<div class="disclaimer note">
<p><strong>Disclaimer:</strong> Some internal fields and implementation details are omitted here for security reasons.</p>
</div>
<h1>Checkpoint Protection System</h1>
<div class="toc">
<h2>Contents</h2>
<ul>
<li><a href="#overview">Overview</a></li>
<li><a href="#how-it-works">How It Works</a></li>
<li><a href="#challenge-generation">Challenge Generation</a></li>
<li><a href="#proof-verification">Proof Verification</a></li>
<li><a href="#token-structure">Token Structure</a></li>
<li><a href="#security-features">Security Features</a></li>
<li><a href="#configuration">Configuration Options</a></li>
<li><a href="#middleware">Middleware Integration</a></li>
<li><a href="#client-side">Client-Side Implementation</a></li>
<li><a href="#api-endpoints">API Endpoints</a></li>
</ul>
</div>
<section id="overview" class="section">
<h2>Overview</h2>
<p>Checkpoint Protection asks visitors to solve a quick puzzle before letting them through, cutting down on automated traffic while keeping the experience smooth for real users.</p>
<ul>
<li>No account or personal data needed</li>
<li>Privacy-focused and lightweight</li>
<li>Blocks bots and scripts effectively</li>
<li>Works seamlessly in modern browsers</li>
</ul>
</section>
<section id="how-it-works" class="section">
<h2>How It Works</h2>
<p>When you navigate to a protected page, the middleware checks for a valid token cookie (<code>__Host-checkpoint_token</code>).</p>
<ol>
<li>If the token is present, the server verifies its signature and confirms it's bound to your device.</li>
<li>Missing or invalid tokens trigger an interstitial page with a request ID.</li>
<li>The browser fetches challenge data from <code>/api/pow/challenge?id=REQUEST_ID</code>. This payload includes a random challenge, salt, difficulty, and hidden parameters.</li>
<li>The client runs two proofs in parallel:
<ul>
<li><strong>Proof of Work:</strong> finds a nonce such that <code>SHA‑256(challenge + salt + nonce)</code> meets the difficulty.</li>
<li><strong>Proof of Space:</strong> allocates and hashes large memory buffers to confirm resource availability.</li>
</ul>
</li>
<li>Results are sent to <code>/api/pow/verify</code> along with the request ID.</li>
<li>On success, the server issues a signed token (valid for 24h) and sets it as a cookie for future visits.</li>
</ol>
<div class="diagram">
<h3>Checkpoint Protection Flow</h3>
<img src="/images/Basic-POW-Overview.excalidraw.svg" alt="Checkpoint Protection Flow Diagram" id="flowDiagram" loading="lazy">
</div>
</section>
<section id="challenge-generation" class="section">
<h2>Challenge Generation</h2>
<p>
Challenges are generated using cryptographically secure random bytes combined with a salt for additional entropy:
</p>
<div class="code-example">
<span class="code-label">Go</span>
<pre><code>func generateChallenge() (string, string) {
// Generate a random challenge
randomBytes := make([]byte, 16)
_, err := cryptorand.Read(randomBytes)
if err != nil {
log.Fatalf("CRITICAL: Failed to generate secure random challenge: %v", err)
}
// Generate a random salt for additional entropy
saltBytes := make([]byte, saltLength)
_, err = cryptorand.Read(saltBytes)
if err != nil {
log.Fatalf("CRITICAL: Failed to generate secure random salt: %v", err)
}
return hex.EncodeToString(randomBytes), hex.EncodeToString(saltBytes)
}</code></pre>
</div>
<div class="note">
<p>
<strong>Security Note:</strong> The system uses Go's crypto/rand package for secure random number generation, ensuring challenges cannot be predicted even by sophisticated attackers.
</p>
</div>
<h3>Challenge Parameters</h3>
<p>
Challenges are stored with a unique request ID and include parameters for verification:
</p>
<div class="code-example">
<span class="code-label">Go</span>
<pre><code>type ChallengeParams struct {
Challenge string `json:"challenge"` // Base64 encoded
Salt string `json:"salt"` // Base64 encoded
Difficulty int `json:"difficulty"`
ExpiresAt time.Time `json:"expires_at"`
ClientIP string `json:"-"`
PoSSeed string `json:"pos_seed"` // Hex encoded
}</code></pre>
</div>
<p>
When a client requests a challenge, the parameters are delivered in an obfuscated format to prevent automated analysis:
</p>
<div class="code-example">
<span class="code-label">JSON</span>
<pre><code>{
"a": "base64-encoded-challenge",
"b": "base64-encoded-salt",
"c": 4,
"d": "hex-encoded-pos-seed"
}</code></pre>
</div>
</section>
<section id="proof-verification" class="section">
<h2>Proof Verification</h2>
<p>
The system performs a two-step verification process:
</p>
<h3>1. Computational Proof (Proof of Work)</h3>
<p>
Verification checks that the hash of the challenge, salt, and nonce combination has the required number of leading zeros:
</p>
<div class="code-example">
<span class="code-label">Go</span>
<pre><code>func verifyProofOfWork(challenge, salt, nonce string, difficulty int) bool {
input := challenge + salt + nonce
hash := calculateHash(input)
// Check if the hash has the required number of leading zeros
prefix := strings.Repeat("0", difficulty)
return strings.HasPrefix(hash, prefix)
}
func calculateHash(input string) string {
hash := sha256.Sum256([]byte(input))
return hex.EncodeToString(hash[:])
}</code></pre>
</div>
<h3>2. Memory Proof (Proof of Space)</h3>
<p>
In addition to the computational work, clients must prove they can allocate and manipulate significant memory resources:
</p>
<ul>
<li>Clients allocate between 48MB to 160MB of memory (size determined by the PoS seed)</li>
<li>Client divides memory into 4-8 chunks and performs deterministic filling operations</li>
<li>The process is run three times, hashing the entire buffer each time</li>
<li>The resulting hashes and execution times are submitted for verification</li>
</ul>
<p>
The server verifies:
</p>
<ul>
<li>All three hashes are identical (proving deterministic execution)</li>
<li>Each hash is 64 characters (valid SHA-256)</li>
<li>Execution times are consistent (within 20% variation)</li>
</ul>
<div class="note">
<p>
The dual-verification approach makes the system resistant to specialized hardware acceleration. While the computational proof can be solved by ASICs or GPUs, the memory proof is specifically designed to be inefficient on such hardware.
</p>
</div>
</section>
<section id="token-structure" class="section">
<h2>Token Structure</h2>
<p>
Checkpoint tokens contain various fields for security and binding:
</p>
<div class="table-container">
<table>
<tr>
<th>Field</th>
<th>Description</th>
<th>Purpose</th>
</tr>
<tr>
<td>Nonce</td>
<td>The solution to the challenge</td>
<td>Verification proof</td>
</tr>
<tr>
<td>ExpiresAt</td>
<td>Token expiration timestamp</td>
<td>Enforces time-limited access (24 hours)</td>
</tr>
<tr>
<td>ClientIP</td>
<td>Hashed full client IP</td>
<td>Device binding (first 8 bytes of SHA-256)</td>
</tr>
<tr>
<td>UserAgent</td>
<td>Hashed user agent</td>
<td>Browser binding</td>
</tr>
<tr>
<td>BrowserHint</td>
<td>Derived from Sec-CH-UA headers</td>
<td>Additional client identity verification</td>
</tr>
<tr>
<td>Entropy</td>
<td>Random data</td>
<td>Prevents token prediction/correlation</td>
</tr>
<tr>
<td>Created</td>
<td>Token creation timestamp</td>
<td>Token age tracking</td>
</tr>
<tr>
<td>LastVerified</td>
<td>Last verification timestamp</td>
<td>Token usage tracking</td>
</tr>
<tr>
<td>Signature</td>
<td>HMAC signature</td>
<td>Prevents token forgery</td>
</tr>
<tr>
<td>TokenFormat</td>
<td>Version number</td>
<td>Backward compatibility support</td>
</tr>
</table>
</div>
<div class="code-example">
<span class="code-label">Go</span>
<pre><code>type CheckpointToken struct {
Nonce string `json:"g"` // Nonce
ExpiresAt time.Time `json:"exp"`
ClientIP string `json:"cip,omitempty"`
UserAgent string `json:"ua,omitempty"`
BrowserHint string `json:"bh,omitempty"`
Entropy string `json:"ent,omitempty"`
Created time.Time `json:"crt"`
LastVerified time.Time `json:"lvf,omitempty"`
Signature string `json:"sig,omitempty"`
TokenFormat int `json:"fmt"`
}</code></pre>
</div>
<h3>Token Security</h3>
<p>
Every token is cryptographically signed using HMAC-SHA256 with a server-side secret:
</p>
<div class="code-example">
<span class="code-label">Go</span>
<pre><code>func computeTokenSignature(token CheckpointToken, tokenBytes []byte) string {
tokenCopy := token
tokenCopy.Signature = "" // Ensure signature field is empty for signing
tokenToSign, _ := json.Marshal(tokenCopy)
h := hmac.New(sha256.New, hmacSecret)
h.Write(tokenToSign)
return hex.EncodeToString(h.Sum(nil))
}
func verifyTokenSignature(token CheckpointToken, tokenBytes []byte) bool {
if token.Signature == "" {
return false
}
expectedSignature := computeTokenSignature(token, tokenBytes)
return hmac.Equal([]byte(token.Signature), []byte(expectedSignature))
}</code></pre>
</div>
<h3>Token Storage</h3>
<p>
Successfully verified tokens are stored in a persistent store for faster validation:
</p>
<div class="code-example">
<span class="code-label">Go</span>
<pre><code>// TokenStore manages persistent storage of verified tokens
type TokenStore struct {
VerifiedTokens map[string]time.Time `json:"verified_tokens"`
Mutex sync.RWMutex `json:"-"`
FilePath string `json:"-"`
}
// Each token is identified by a unique hash
func calculateTokenHash(token CheckpointToken) string {
data := fmt.Sprintf("%s:%s:%d",
token.Nonce, // Use nonce as part of the key
token.Entropy, // Use entropy as part of the key
token.Created.UnixNano()) // Use creation time
hash := sha256.Sum256([]byte(data))
return hex.EncodeToString(hash[:])
}</code></pre>
</div>
</section>
<section id="security-features" class="section">
<h2>Security Features</h2>
<div class="security">
<h3>Anti-Forgery Protections</h3>
<ul>
<li><strong>HMAC Signatures:</strong> Each token is cryptographically signed using HMAC-SHA256 to prevent tampering</li>
<li><strong>Token Binding:</strong> Tokens are bound to client properties (hashed full IP, hashed user agent, browser client hints)</li>
<li><strong>Random Entropy:</strong> Each token contains unique entropy to prevent token prediction or correlation</li>
<li><strong>Format Versioning:</strong> Tokens include a format version to support evolving security requirements</li>
</ul>
</div>
<div class="security">
<h3>Replay Prevention</h3>
<ul>
<li><strong>Nonce Tracking:</strong> Used nonces are tracked to prevent replay attacks</li>
<li><strong>Expiration Times:</strong> All tokens and challenges have expiration times</li>
<li><strong>Token Cleanup:</strong> Expired tokens are automatically purged from the system</li>
<li><strong>Challenge Invalidation:</strong> Challenges are immediately invalidated after successful verification</li>
</ul>
</div>
<div class="security">
<h3>Rate Limiting</h3>
<ul>
<li><strong>IP-Based Limits:</strong> Maximum verification attempts per hour (default: 10)</li>
<li><strong>Request ID Binding:</strong> Challenge parameters are bound to the requesting IP</li>
<li><strong>Challenge Expiration:</strong> Challenges expire after 5 minutes to prevent stockpiling</li>
</ul>
</div>
<div class="security">
<h3>Advanced Verification</h3>
<ul>
<li><strong>Proof of Space:</strong> Memory-intensive operations prevent GPU/ASIC acceleration</li>
<li><strong>Browser Fingerprinting:</strong> Secure client-hint headers verify legitimate browsers</li>
<li><strong>Challenge Obfuscation:</strong> Challenges are encoded and structured to resist automated analysis</li>
<li><strong>Persistent Secret:</strong> The system uses a persistent HMAC secret stored securely on disk</li>
</ul>
</div>
</section>
<section id="configuration" class="section">
<h2>Configuration Options</h2>
<p>
The Checkpoint system can be configured through these constants:
</p>
<div class="table-container">
<table>
<tr>
<th>Constant</th>
<th>Description</th>
<th>Default</th>
</tr>
<tr>
<td>Difficulty</td>
<td>Number of leading zeros required in the hash</td>
<td>4</td>
</tr>
<tr>
<td>TokenExpiration</td>
<td>Duration for which a token is valid</td>
<td>24 hours</td>
</tr>
<tr>
<td>Cookie Name</td>
<td>__Host-checkpoint_token</td>
<td>The cookie name storing the issued token</td>
</tr>
<tr>
<td>maxAttemptsPerHour</td>
<td>Rate limit for verification attempts</td>
<td>10</td>
</tr>
<tr>
<td>saltLength</td>
<td>Length of the random salt in bytes</td>
<td>16</td>
</tr>
<tr>
<td>maxNonceAge</td>
<td>Time before nonces are cleaned up</td>
<td>24 hours</td>
</tr>
<tr>
<td>challengeExpiration</td>
<td>Time before a challenge expires</td>
<td>5 minutes</td>
</tr>
</table>
</div>
<div class="warning">
<p>
<strong>Warning:</strong> Increasing the Difficulty significantly increases the computational work required by clients.
A value that's too high may result in poor user experience, especially on mobile devices.
</p>
</div>
<div class="code-example">
<span class="code-label">Go</span>
<pre><code>const (
// Difficulty defines the number of leading zeros required in hash
Difficulty = 4
// TokenExpiration sets token validity period
TokenExpiration = 24 * time.Hour
// CookieName defines the cookie name for tokens
CookieName = "__Host-checkpoint_token"
// Max verification attempts per IP per hour
maxAttemptsPerHour = 10
// Salt length for additional entropy
saltLength = 16
)</code></pre>
</div>
</section>
<section id="middleware" class="section">
<h2>Middleware Integration</h2>
<p>
The Checkpoint system provides a middleware handler that automatically protects HTML routes while bypassing API routes and static assets:
</p>
<h3>HTMLCheckpointMiddleware</h3>
<p>
This middleware is optimized for HTML routes, with smart content-type detection and automatic exclusions for static assets and API endpoints.
</p>
<div class="code-example">
<span class="code-label">Go</span>
<pre><code>// HTMLCheckpointMiddleware handles challenges specifically for HTML pages
func HTMLCheckpointMiddleware() fiber.Handler {
return func(c *fiber.Ctx) error {
// Allow certain paths to bypass verification
path := c.Path()
if path == "/video-player" || path == "/video-player.html" || strings.HasPrefix(path, "/videos/") {
return c.Next()
}
if strings.HasPrefix(path, "/api") {
return c.Next()
}
if path == "/favicon.ico" || (strings.Contains(path, ".") && !strings.HasSuffix(path, ".html")) {
return c.Next()
}
// Only apply to HTML routes
isHtmlRoute := strings.HasSuffix(path, ".html") || path == "/" ||
(len(path) > 0 && !strings.Contains(path, "."))
if !isHtmlRoute {
return c.Next()
}
token := c.Cookies(CookieName)
if token != "" {
valid, err := validateToken(token, c)
if err == nil && valid {
return c.Next()
}
}
return serveInterstitial(c)
}
}</code></pre>
</div>
<h3>Usage in Application</h3>
<div class="code-example">
<span class="code-label">Go</span>
<pre><code>// Enable HTML checkpoint protection for all routes
app.Use(middleware.HTMLCheckpointMiddleware())
// API group with verification endpoints
api := app.Group("/api")
// Verification endpoints
api.Post("/pow/verify", middleware.VerifyCheckpointHandler)
api.Get("/pow/challenge", middleware.GetCheckpointChallengeHandler)
// Example protected API endpoint
api.Get("/protected", func(c *fiber.Ctx) error {
// Access is already verified by cookie presence
return c.JSON(fiber.Map{
"message": "You have accessed the protected endpoint!",
"time": time.Now(),
})
})</code></pre>
</div>
</section>
<section id="client-side" class="section">
<h2>Client-Side Implementation</h2>
<p>
The client-side implementation is handled by the interstitial page and its associated JavaScript:
</p>
<ol>
<li>Client attempts to access a protected resource</li>
<li>Server serves the interstitial page with a request ID</li>
<li>JavaScript fetches challenge parameters from <code>/api/pow/challenge?id=REQUEST_ID</code></li>
<li>Two verification stages run in parallel:
<ul>
<li>Computational proof: Using Web Workers to find a valid nonce</li>
<li>Memory proof: Allocating and manipulating memory buffers</li>
</ul>
</li>
<li>Results are submitted to <code>/api/pow/verify</code> endpoint</li>
<li>On success, the server sets a cookie and redirects to the original URL</li>
</ol>
<h3>Web Worker Implementation</h3>
<p>
Computational proof is handled by Web Workers to avoid freezing the UI:
</p>
<div class="code-example">
<span class="code-label">JavaScript</span>
<pre><code>function workerFunction() {
self.onmessage = function(e) {
const { type, data } = e.data;
if (type === 'pow') {
// PoW calculation
const { challenge, salt, startNonce, endNonce, target, batchId } = data;
let count = 0;
let solution = null;
processNextNonce(startNonce);
function processNextNonce(nonce) {
const input = String(challenge) + String(salt) + nonce.toString();
const msgBuffer = new TextEncoder().encode(input);
crypto.subtle.digest('SHA-256', msgBuffer)
.then(hashBuffer => {
const hashArray = Array.from(new Uint8Array(hashBuffer));
const result = hashArray.map(b =>
b.toString(16).padStart(2, '0')).join('');
count++;
if (result.startsWith(target)) {
solution = { nonce: nonce.toString(), found: true };
self.postMessage({
type: 'pow_result',
solution: solution,
count: count,
batchId: batchId
});
return;
}
if (nonce < endNonce && !solution) {
setTimeout(() => processNextNonce(nonce + 1), 0);
} else if (!solution) {
self.postMessage({
type: 'pow_result',
solution: null,
count: count,
batchId: batchId
});
}
});
}
}
};
}</code></pre>
</div>
<h3>Memory Proof Implementation</h3>
<p>
The memory proof allocates and manipulates large buffers to verify client capabilities:
</p>
<div class="code-example">
<span class="code-label">JavaScript</span>
<pre><code>async function runProofOfSpace(seedHex, isDecoy) {
// Deterministic memory size (48MB to 160MB) based on seed
const minMB = 48, maxMB = 160;
let seedInt = parseInt(seedHex.slice(0, 8), 16);
const CHUNK_MB = minMB + (seedInt % (maxMB - minMB + 1));
const CHUNK_SIZE = CHUNK_MB * 1024 * 1024;
// Chunk memory for controlled allocation
const chunkCount = 4 + (seedInt % 5); // 4-8 chunks
const chunkSize = Math.floor(CHUNK_SIZE / chunkCount);
// Run the proof multiple times to verify consistency
const runs = 3;
const hashes = [];
const times = [];
// For each run...
for (let r = 0; r < runs; r++) {
// Generate deterministic chunk order
let prng = seededPRNG(seedHex + r.toString(16));
let order = Array.from({length: chunkCount}, (_, i) => i);
for (let i = order.length - 1; i > 0; i--) {
const j = prng() % (i + 1);
[order[i], order[j]] = [order[j], order[i]];
}
// Allocate and fill memory buffer
let t0 = performance.now();
let buf = new ArrayBuffer(CHUNK_SIZE);
let view = new Uint8Array(buf);
// Fill buffer with deterministic pattern
for (let c = 0; c < chunkCount; c++) {
let chunkIdx = order[c];
let start = chunkIdx * chunkSize;
let end = (chunkIdx + 1) * chunkSize;
for (let i = start; i < end; i += 4096) {
view[i] = prng() & 0xFF;
}
}
// Hash the entire buffer
let hashBuf = await crypto.subtle.digest('SHA-256', view);
let t2 = performance.now();
// Convert hash to hex string
let hashHex = Array.from(new Uint8Array(hashBuf))
.map(b => b.toString(16).padStart(2, '0')).join('');
// Store results
hashes.push(hashHex);
times.push(Math.round(t2 - t0));
// Clean up
buf = null; view = null;
}
return { hashes, times };
}</code></pre>
</div>
<div class="note">
<p>
The client-side implementation is designed to be difficult to reverse-engineer. The obfuscated API responses, minimal logging, and anti-debugging measures prevent automated circumvention.
</p>
</div>
</section>
<section id="api-endpoints" class="section">
<h2>API Endpoints</h2>
<p>
The Checkpoint system exposes two primary API endpoints:
</p>
<h3>1. Challenge Endpoint</h3>
<p>
Retrieves challenge parameters for a verification request:
</p>
<div class="code-example">
<span class="code-label">HTTP</span>
<pre><code>GET /api/pow/challenge?id=REQUEST_ID
Response:
{
"a": "base64-encoded-challenge",
"b": "base64-encoded-salt",
"c": 4,
"d": "hex-encoded-pos-seed"
}</code></pre>
</div>
<h3>2. Verification Endpoint</h3>
<p>
Accepts proof solutions and issues tokens when valid:
</p>
<div class="code-example">
<span class="code-label">HTTP</span>
<pre><code>POST /api/pow/verify
Request:
{
"request_id": "unique-request-id",
"g": "nonce-solution",
"h": ["pos-hash1", "pos-hash2", "pos-hash3"],
"i": [time1, time2, time3]
}
Response:
{
"token": "base64-encoded-token",
"expires_at": "2025-04-17T18:57:48Z"
}</code></pre>
</div>
<div class="note">
<p>
<strong>Backwards Compatibility:</strong> The older endpoint <code>/api/verify</code> is maintained for compatibility with existing clients.
</p>
</div>
</section>
<footer>
<div class="doc-version-note">These docs reflect version 2.0 of the <strong>Checkpoint Protection System</strong>.</div>
<p>Last updated: <span id="last-updated">Tuesday, April 16, 2025</span></p>
</footer>
</div>
</body>
</html>