Fluid URL API

URL Parsing and Manipulation

The URL interface provides comprehensive URL parsing, encoding, and manipulation capabilities similar to Python's urllib.parse module. It supports RFC 3986 compliant URL handling with features for encoding/decoding, query string manipulation, URL joining, and component extraction.

The module is designed for HTTP client/server development, web API integration, and any application requiring robust URL manipulation.

It can be loaded with the line:

url = require('net/url')

URL Encoding Functions

url.encode()

encoded = url.encode(String, ExtraChars)

Percent-encodes a string for safe URL use, encoding reserved characters according to RFC 3986.

encoded = url.encode('hello world!')
-- Result: 'hello%20world%21'

encoded = url.encode('user@domain.com')
-- Result: 'user%40domain.com'

The optional ExtraChars parameter allows additional characters to be encoded beyond the standard reserved set.

url.encodePlus()

encoded = url.encodePlus(String, ExtraChars)

Percent-encodes a string with spaces converted to plus signs, suitable for form data encoding.

encoded = url.encodePlus('hello world')
-- Result: 'hello+world'

url.decode()

decoded = url.decode(String)

Decodes a percent-encoded string back to its original form.

decoded = url.decode('hello%20world%21')
-- Result: 'hello world!'

url.decodePlus()

decoded = url.decodePlus(String)

Decodes a percent-encoded string with plus signs converted to spaces.

decoded = url.decodePlus('hello+world')
-- Result: 'hello world'

url.encodeComponent()

encoded = url.encodeComponent(String)

Encodes a string for safe use in URL path components.

encoded = url.encodeComponent('file name.txt')
-- Result: 'file%20name.txt'

Query String Functions

url.parseQuery()

params = url.parseQuery(QueryString)

Parses a query string into a table of key-value pairs. Duplicate keys are automatically converted to arrays.

params = url.parseQuery('name=John&age=30&tag=red&tag=blue')
-- Result: { name = 'John', age = '30', tag = { 'red', 'blue' } }

url.parseQueryList()

params = url.parseQueryList(QueryString)

Parses a query string into an ordered list of key-value pairs, preserving parameter order and duplicates.

params = url.parseQueryList('first=1&second=2&first=3')
-- Result: {
--   { key = 'first', value = '1' },
--   { key = 'second', value = '2' },
--   { key = 'first', value = '3' }
-- }

url.buildQuery()

queryString = url.buildQuery(Params)

Builds a query string from a table or list of parameters.

params = { name = 'John Doe', age = 30 }
query = url.buildQuery(params)
-- Result: 'name=John+Doe&age=30'

list = { { key = 'first', value = '1' }, { key = 'second', value = '2' } }
query = url.buildQuery(list)
-- Result: 'first=1&second=2'

url.encodeParams()

queryString = url.encodeParams(Params)

Alias for url.buildQuery() - encodes parameters into query string format.

URL Parsing Functions

url.parse()

components = url.parse(UrlString)

Parses a URL into its component parts, returning a table with the following fields:

parts = url.parse('https://user:pass@example.com:8080/path?query=value#section')
-- Result: {
--   scheme = 'https',
--   auth = 'user:pass',
--   host = 'example.com',
--   port = 8080,
--   path = '/path',
--   query = 'query=value',
--   fragment = 'section'
-- }

url.unparse()

urlString = url.unparse(Components)

Reconstructs a URL from component parts.

components = {
   scheme = 'https',
   host = 'example.com',
   path = '/api/users',
   query = 'limit=10'
}
url = url.unparse(components)
-- Result: 'https://example.com/api/users?limit=10'

url.split()

components = url.split(UrlString)

Splits a URL into 5 basic components (scheme, netloc, path, query, fragment) similar to Python's urlsplit.

parts = url.split('https://example.com:8080/path?query=value#section')
-- Result: {
--   scheme = 'https',
--   netloc = 'example.com:8080',
--   path = '/path',
--   query = 'query=value',
--   fragment = 'section'
-- }

url.unsplit()

urlString = url.unsplit(Components)

Reconstructs a URL from split components.

url.defrag()

base, fragment = url.defrag(UrlString)

Removes and returns the fragment portion of a URL.

base, frag = url.defrag('https://example.com/path#section')
-- base = 'https://example.com/path'
-- frag = 'section'

URL Manipulation Functions

url.join()

joinedUrl = url.join(BaseUrl, RelativeUrl)

Joins a base URL with a relative URL, handling path resolution and normalisation.

joined = url.join('https://example.com/api/', 'users/123')
-- Result: 'https://example.com/api/users/123'

joined = url.join('https://example.com/api/v1/', '../v2/users')
-- Result: 'https://example.com/api/v2/users'

url.normalize()

normalizedPath = url.normalize(Path)

Normalises a URL path by resolving . and .. components.

normalized = url.normalize('/api/v1/../v2/./users')
-- Result: '/api/v2/users'

Utility Functions

url.isAbsolute()

isAbs = url.isAbsolute(UrlString)

Returns true if the URL is absolute (contains a scheme).

isAbs = url.isAbsolute('https://example.com')  -- true
isAbs = url.isAbsolute('/relative/path')       -- false

url.isRelative()

isRel = url.isRelative(UrlString)

Returns true if the URL is relative (no scheme).

url.getScheme()

scheme = url.getScheme(UrlString)

Extracts and returns the scheme from a URL.

scheme = url.getScheme('https://example.com')
-- Result: 'https'

url.getHost()

host = url.getHost(UrlString)

Extracts and returns the hostname from a URL.

host = url.getHost('https://example.com:8080/path')
-- Result: 'example.com'

url.getPort()

port = url.getPort(UrlString, DefaultPort)

Extracts the port number from a URL, optionally returning a default if no port is specified.

port = url.getPort('https://example.com:8080')     -- 8080
port = url.getPort('https://example.com')          -- nil
port = url.getPort('https://example.com', 443)     -- 443

Special Features

IPv6 Support

The URL parser fully supports IPv6 addresses in URLs:

parts = url.parse('http://[2001:db8::1]:8080/path')
-- parts.host = '2001:db8::1'
-- parts.port = 8080

Error Handling

URL parsing functions return nil for invalid input:

parts = url.parse('')      -- returns nil
parts = url.parse(nil)     -- returns nil

RFC 3986 Compliance

The module follows RFC 3986 standards for URL parsing and encoding, ensuring proper handling of reserved characters and edge cases.

Usage Examples

Building API URLs

base = 'https://api.example.com/v1/'
endpoint = url.join(base, 'users/search')
params = { q = 'john doe', limit = 10, active = true }
query = url.buildQuery(params)
apiUrl = endpoint .. '?' .. query
-- Result: 'https://api.example.com/v1/users/search?q=john+doe&limit=10&active=true'

Parsing Request URLs

requestUrl = 'https://example.com/api/users?name=John&age=30#profile'
parts = url.parse(requestUrl)
queryParams = url.parseQuery(parts.query)

print('Host:', parts.host)           -- example.com
print('Path:', parts.path)           -- /api/users
print('Name:', queryParams.name)     -- John
print('Fragment:', parts.fragment)   -- profile

Safe URL Construction

userInput = 'file with spaces & special chars!'
safePath = url.encodeComponent(userInput)
fullUrl = url.join('https://cdn.example.com/files/', safePath)
-- Result: 'https://cdn.example.com/files/file%20with%20spaces%20%26%20special%20chars%21'