Your PCAP and Your SIEM Timestamps Disagree. Whose Clock Wins?

Packet capture, firewall, EDR and SIEM timestamps showing four different times for one incident

When a packet capture, a firewall log and a SIEM alert describe the same incident, their timestamps rarely agree. Here is what that looks like. The timeline below is constructed for illustration, not taken from a real case, but anyone who has sat through an incident review will recognize the shape of it.

SystemReported timeWhere the time comes from
Packet capture (GPS-disciplined)14:02:07.331Hardware timestamp at the wire
Firewall (NTP, drifting)14:02:09Drifting clock, one-second resolution
EDR agent14:01:58Endpoint clock nobody has checked
SIEM14:02:11Time the record arrived, not the time of the event

Four systems, one event, thirteen seconds of disagreement. Now ask the question the review actually turns on: which came first, the process or the connection? Read the EDR against the capture and the process started nine seconds before the connection. But nobody has checked the endpoint’s clock. If it runs ten seconds slow, the process started after the connection, and the story of the incident reverses.

For Canadian security teams operating across regulated sectors—from energy utilities and financial institutions to telecommunications and critical infrastructure—this ambiguity is more than an annoyance. With regulatory frameworks tightening and boards demanding defensible incident reporting, knowing when an event occurred is just as critical as knowing what happened. Yet, across distributed enterprise environments, multi-vendor tool sprawl means your packet capture, firewall, and SIEM are often speaking entirely different temporal dialects.

We have covered why timestamps matter for packet capture before. This post starts one step later: the capture clock is only one of several clocks in the investigation. What follows is why those clocks disagree, why a nanosecond figure on a data sheet does not settle the argument, and what a defensible fix looks like.

Five reasons the timestamps disagree

Every system that produces evidence stamps it from its own clock, using its own sync source, at its own point in the data path. NIST’s Guide to Computer Security Log Management puts it plainly: each host references its internal clock when it stamps a log entry, and if that clock is inaccurate, the timestamps are too. In practice the disagreement comes from five places.

  1. Different sync sources. One device follows GNSS, another PTP, most follow NTP, and a few follow nothing at all.
  2. Drift between syncs. A clock is corrected only when it hears from its source. In between it runs on its own oscillator, and when the source drops it carries on with whatever holdover it has, which is sometimes none.
  3. Event time versus ingest time. A log record usually carries two times: when the event happened and when the collector received it. If a view sorts on arrival, queueing and batching delays quietly reorder the story.
  4. Formats, time zones and precision. Seconds sit beside milliseconds and nanoseconds, local time beside UTC. Truncation alone can move an event by most of a second.
  5. Different points in the data path. The wire, the packet broker, the sensor and the log pipeline each see the same packet at a different moment.

None of this usually breaks anything, which is why it goes unnoticed. Security infrastructure is built to tolerate skew. Kerberos, for example, accepts a default difference of five minutes between a client clock and a domain controller. A clock that is ninety seconds out still authenticates. Tolerable for login is not good enough for forensics.

Resolution is not accuracy

A timestamp has two properties that data sheets tend to blur. Resolution is the size of the tick: how finely the clock divides time. Accuracy is the offset from true time: how far the clock sits from UTC. The two are independent.

A kitchen scale that reads to a tenth of a gram can still be ten grams out if nobody has zeroed it. Clocks work the same way. A capture device may stamp every packet from a counter that ticks every 8 nanoseconds. If the clock behind that counter is set by NTP and sits a millisecond from UTC, every one of those timestamps is a millisecond wrong, reported to nine decimal places.

On that one device the spacing between packets is still excellent. The trouble starts when you lay its capture beside evidence from another device with a different error. When a data sheet says “nanosecond timestamps,” it is almost always describing resolution.

The property that matters for evidence is traceability to UTC, which is less exotic than it sounds. It comes down to two questions: who disciplines this clock, and how do you know it is right today? If you can name the source, show the chain from that source to the device, and produce a record of the measured offset, the timestamp is traceable. If the answer is “it’s on NTP, I think,” it is a number with no error bar.

Where the timestamp is applied

The same packet can be stamped at the TAP or capture card, on the packet broker, in the sensor, or only when a log line is written. Each step away from the wire adds delay, and under load that delay varies. It is one reason how you access traffic is a design decision and not a cabling detail.

Some devices carry the timestamp inside the packet itself, in a trailer, an added header or a field of a mirroring protocol, so that it survives the trip through the packet broker layer to the tool. That only works if the consuming tool understands the format. If it does not, it may stamp the packet on arrival and discard the hardware timestamp you paid for.

We raise this as a question to ask, not a fault to assume. For every hop between the wire and the analyst’s screen: where is the timestamp applied, in what format, and does the next device read it or replace it? It is quick to prove in an evaluation, and better proven before an incident than during one.

Three ways to discipline a clock

GNSS with a pulse-per-second output. A receiver takes time directly from satellites. The US Naval Observatory states that GPS users can obtain time transfer to UTC(USNO) within 40 nanoseconds, 95 percent of the time, and NIST notes that multipath errors alone can add tens of nanoseconds at the receiver. The cost is an antenna with a clear view of the sky, a cable run to reach it, and exposure to interference. It fits the grandmaster, and the capture points whose timeline will be argued over.

PTP (IEEE 1588). PTP distributes time across the network from a grandmaster, with timestamps taken in hardware. The standard was written for sub-microsecond synchronization. The cost is network support: switches that act as boundary or transparent clocks, and a grandmaster worth following. Without them accuracy degrades quietly. Our PTP explainer covers the moving parts.

NTP. NTP runs on everything and costs almost nothing. NIST researchers found that path asymmetry typically limits NTP over wide-area networks to about a millisecond; a well-run local network does better. That is adequate for many log sources. It is the wrong tool for stamping packets that arrive microseconds apart.

None of the three is the right answer everywhere. Most sound designs use all of them: GNSS at the top, PTP to the devices that need it, NTP to everything else, and one source they all trace back to.

When time becomes the attack

Time is also an attack surface. If an attacker can shift a clock or spoof its source, evidence stops lining up, and anything that depends on time, from certificate validity to the correlation windows in detection rules, can fail in ways that look like ordinary faults. CISA’s time guidance for network operators draws the same link between time offsets and the ability to reconstruct an intrusion timeline.

Two exposures are worth knowing about. GNSS signals are weak by the time they reach the ground and can be jammed or spoofed; CISA publishes guidance on resilient timing for exactly that reason. And classic NTP and PTP exchanges carry no cryptographic proof of where they came from.

Authenticated options are arriving. Network Time Security for NTP was standardized as RFC 8915 in 2020. An equivalent for PTP is an active IETF draft, still a work in progress. Until both are routine, the practical defence is the one a good timing design gives you anyway: more than one source, holdover, and monitoring that notices when clocks stop agreeing. Detecting attacks on time is a subject of its own, and beyond this post.

The Regulatory Cost of Drift: Bill C-8 and Audit Readiness

In Canada, the regulatory landscape is shifting to demand higher standards of operational visibility and cyber resilience. Under the Critical Cyber Systems Protection Act (CCSPA / Bill C-8), designated operators are required to report cyber incidents swiftly and maintain rigorous records of system activity.

When regulators or forensic investigators examine an intrusion timeline, “approximate time” does not suffice. If your logs cannot prove traceability back to UTC—with documented error bars and synchronized timestamps across your security stack—your incident reconstruction becomes legally and operationally vulnerable. A timeline skewed by even a few seconds can obscure lateral movement, invalidate breach attribution, and complicate mandatory reporting compliance. Traceable time is no longer just a technical best practice; it is a compliance requirement.

A time contract for your evidence

The fix is not a better tool. It is a deliberate timing design, written down. We think of it as a time contract: a short document that states what every evidence source promises about its clock.

  • Name an authoritative source, and name the person who owns it.
  • Discipline every evidence producer from it, or record why not.
  • Record each source’s sync method and estimated uncertainty. “NTP, within a few milliseconds” is a perfectly good entry. An empty cell is the problem.
  • Monitor offsets and alarm on drift. A clock that was right at commissioning proves nothing about today.
  • Store event time in UTC, in ISO 8601 format, and keep ingest time as a separate field. Never let one overwrite the other.
  • Re-test after changes. Firmware, topology and tool upgrades all move clocks.

None of this requires replacing the tools you have. It requires knowing what each one promises, which is also the starting point for deciding where better timing would actually change an outcome.

Five questions to ask your team on Monday

What time source disciplines each of your capture points?

If the answer differs by site, or nobody is sure, that is the first entry in the contract.

Does your SIEM sort on event time or ingest time?

Check the views your analysts actually use, not only the default.

What is the worst clock offset in your security stack today, and who would notice?

If nothing measures it, the honest answer is “unknown.”

If the time source dropped for four hours, what would your timestamps be worth?

This is a holdover question, and it has a different answer for every device.

Could you show an auditor how each timestamp traces back to UTC?

Not every source needs nanoseconds. Every source needs an answer.

Where to start

If those questions were harder to answer than expected, that is normal. Timing, visibility and security tooling are usually bought by different teams, in different years, and nobody owns the clock that connects them. We work across all three. 

Fixing this does not mean ripping out your existing security stack. As a Canadian value-added reseller and technical advisor, Telnet Networks works with organizations across the country to bridge the gap between visibility, precision timing, and network security. Whether you need to deploy resilient, dual-source GNSS grandmasters (such as Safran timing solutions) at primary data centers, implement software-based time distribution (like Timebeat) across virtualized environments, or audit your existing NTP and PTP hierarchies, we help you design a timing architecture you can actually defend in an audit or an incident review.

Bring us the list of your evidence sources and we will help you work out which clocks you can trust and which need attention. Talk to our team.


Telnet Networks is a Canadian value-added reseller specializing in network visibility, precision timing and network security. We work with several vendors in each of those areas, and our recommendations depend on the network in front of us.

Unparalleled Efficiency At Scale With SecureSync In Data Operations

Introduction

In the era of big data and distributed systems, achieving efficiency at scale is crucial for data operators. The Safran SecureSync emerges as a game-changing solution, providing unparalleled efficiency and reliability in time synchronization for data operations. We will explore the technical details behind the SecureSync and demonstrate how data operators can gain significant efficiencies at scale by leveraging its advanced features.

Exceptional Precision and Reliability

The SecureSync boasts exceptional precision and reliability, ensuring accurate time synchronization across distributed systems. Its innovative architecture combines precision timing components and advanced technologies, delivering ultra-low phase noise and frequency accuracy. With a holdover stability of <1 µs/day, the SecureSync guarantees uninterrupted synchronization, even in the event of temporary loss of reference signals.

High Scalability and Flexibility

Data operators often deal with expanding infrastructures and evolving requirements. The SecureSync is designed to address these scalability challenges. It supports a high number of simultaneous network clients, accommodating large-scale distributed systems effortlessly. Whether deployed in a small cluster or a global network, the SecureSync seamlessly integrates with existing infrastructure, providing precise time synchronization across all nodes.

Robust Timing Redundancy

The SecureSync ensures reliability in demanding operational environments through its timing redundancy capabilities. It incorporates dual-redundant power supplies and accepts multiple timing sources, minimizing the risk of single points of failure. Redundant timing sources and power supplies guarantee continuous synchronization and prevent disruptions that could impact data operations.

Advanced Network Time Protocol (NTP) and Precision Time Protocol (PTP) Support

SecureSync supports both NTP and PTP, enabling compatibility with a wide range of distributed systems. NTP provides accurate time synchronization for applications that require millisecond-level accuracy, while PTP offers sub-microsecond synchronization for applications with stringent timing requirements. The SecureSync’s ability to support both protocols ensures flexibility in integrating with various data operations, optimizing performance and efficiency.

Compliance and Traceability

Data operators often face stringent compliance requirements and the need for traceability in their operations. The SecureSync addresses these concerns by adhering to industry standards for time synchronization. It provides traceable and auditable event timestamps, facilitating compliance with regulatory frameworks and simplifying the audit process for data operations.

Comprehensive Management and Monitoring Capabilities

To efficiently manage and monitor distributed systems, the SecureSync offers advanced management and monitoring features. Its intuitive web-based interface allows for centralized control, configuration, and monitoring of multiple SecureSync units. The interface provides real-time status updates, performance metrics, and alerts, ensuring proactive management and facilitating rapid troubleshooting.

Conclusion

The SecureSync from Safran empowers data operators to achieve unparalleled efficiency at scale in their distributed systems. With exceptional precision, scalability, redundancy, protocol support, compliance adherence, and comprehensive management capabilities, the SecureSync proves to be a reliable and efficient solution for accurate time synchronization.

By leveraging the technical capabilities of the SecureSync, data operators can ensure seamless data operations, mitigate risks of inconsistencies, and optimize performance at scale. With its advanced features and robust design, the SecureSync emerges as a key enabler for data operators seeking efficiency, reliability, and compliance in their distributed data environments.

Mitigating an NTP Distributed Denial of Service (DDoS) Attack

StableNet Network Management Solutions 7

Who Should Read This White Paper?

  • Network and System Engineers
  • Network and System Architects
  • Network and System Administrators
  • Directors/Managers of IT Infrastructure
  • CTOs

By Pritam Kandel

Introduction

Network time service is not something many businesses think about as a key component of their critical infrastructures. In fact, it is often overlooked entirely, and in error. As a result, the network architect or engineer often defaults to an easy alternative: using a server or network switch as the source of the network clock and synchronizing these sources to Internet time servers using Network Time Protocol (NTP).

However, is the “NTP over Internet” really a secure method to solve network timekeeping requirements? Is it okay for some industries, and not others? Let’s explore the subject.

NTP Over Internet: How Safe is It?

NTP, one of the oldest internet protocols in use, is the standard for synchronizing clocks between computers over a packet-switched network – such as the Internet.

According to the Akamai global state of the Internet security report (Summer 2018), NTP over Internet is the second most common protocol being attacked by DDoS. And, in just a year, DDoS attacks have increased by 16%.

Figure 1: A typical deployment of enterprise timekeeping using NTP over Internet.

Figure 1 shows how a typical NTP over Internet setup works. It consists of a public pool of NTP servers (NTP Stratum 1) that is used as a reference by internal time servers to receive time. This approach requires a communication path between the Internet and internal time servers through the firewall, which opens access to the network and creates a vulnerability that hackers can use to infiltrate your entire system. For networks using this method, not only can the timing infrastructure become ripe for cyberattacks, the quality of time is also compromised, in terms of both precision and accuracy.

A Better Solution: Your Own Stratum 1 NTP Server

If you are using the Internet as the source of your time, it is unfortunately a myth to believe that your firewall — even the next-gen firewall that comes with IDS (Intrusion Detection System) or IPS (Intrusion Prevention System) functionality – will protect you from DDoS attacks.

So how can you mitigate DDoS for time service?

The expression, “a chain is only as strong as its weakest link,” couldn’t be truer in the case of DDoS attacks. At Orolia, we recommend a very effective and simple solution to our customers: Eliminate the weakest link in the chain. In other words, don’t rely on the Internet for your network time.
Figure 2. A typical resilient timing infrastructure for enterprise with Orolia time servers.

Figure 2 shows how an enterprise can eliminate the “weakest link” – by building its own resilient and redundant network timekeeping infrastructure internally, using Orolia time servers, such as the SecureSync.

Each Orolia time server receives time signals through a GNSS (Global Navigation Satellite System) or GPS (Global Position System) antenna and regulates its internal high-quality oscillator clock with that information. The time, with accuracy under Nano seconds, is then distributed to the network. If NTP is used as the preferred protocol, then the server will operate at NTP Stratum level 1, and distribute safe, reliable time to the remainder of your network without the use of an internet connection.

Other Advantages of Internal Timekeeping

In addition to mitigating danger from DDoS attacks, time servers such as those from Orolia offer several other advantages, including:

  1. Resiliency – Each Orolia time server unit can use multi-GNSS for its time reference. However, if a GNSS/GPS signal is not available, Orolia time servers also contain an internal holdover oscillator capable of maintaining accurate time for days, or even months, using atomic clock technology in the absence of a valid GNSS signal.
  2. RF Signal Security – Anti-jamming, anti-spoofing and signal security are engrained in Orolia time servers. Customers in need of even higher levels of security also include our Broadshield™ and anti-jamming antenna solutions.
  3. High Integrity UTC Traceable Time – Sophisticated threats can spoof GNSS. Though these threats are detectable by Broadshield, how is traceability to UTC maintained? STL is there for the rescue. As an alternate encrypted antenna signal, STL supplies powerful authentication to confirm that you have true UTC traceability.
  4. Ease of Installation – Does your environment make it difficult to achieve roof access to capture a GNSS signal? Again, STL to the rescue. Much stronger than GPS or any GNSS signal, STL can be received indoors. At a recent demonstration, STL provided solid reception inside the NYSE building, located in one of the most severe urban canyons in the world, where a view of the sky for GNSS reception is very limited.
  5. Multiple Options – In addition to full NTP compatibility, Orolia time servers support multiple protocols and options to distribute time, like Precision Time Protocol (PTP), Pulse Per Second (PPS) and other time signals as suited to customer requirements. Plus, industry-leading support is standard.

Conclusion

In today’s threat-laden environment, it is only too easy to jam or spoof the network, causing anything from minor disruption to extreme havoc within a critical infrastructure. Reliance upon NTP over Internet has inherent risks, which can easily be mitigated by using your own Stratum 1 NTP server, which will provide high-integrity UTC traceable time. Adding anti-jam and anti-spoof software and antennas will give you an even higher level of resiliency and security. The real question to ask yourself is: Can your company afford the risk of a DDoS attack? If the answer is no, then an upgrade to a Stratum 1 NTP server should be de rigueur.

NTP vs. SNTP: What’s the Difference?

Network Instruments Accurate Monitoring

By David Sohn, Solution Architect

(And Which One Do You Really Need?)

NTP (Network Time Protocol) and SNTP (Simple Network Time Protocol) are similar TCP/IP protocols in that they use the same time packet from a Time Server message to compute accurate time. The procedure used by the Time Server to assemble and send out a time stamp is exactly the same whether NTP (i.e., full implementation NTP) is used, or SNTP is used.

The difference between NTP and SNTP is important in the time synchronization program running on the client side on each system.

The time synchronization program, whether it is a Windows built-in program like W32Time (which uses the SNTP protocol) or a third-party add-on, determines which protocol is being used — not the time server. The time server does not care. The difference between NTP and SNTP is in the error checking and the algorithm for the actual correction to the time itself.

The NTP algorithm is much more complicated than the SNTP algorithm. NTP normally uses multiple time servers to verify the time and then controls the slew rate of the system. The algorithm determines if the values are accurate using several methods, including fudge factors and identifying time servers that don’t agree with the other time servers. It then speeds up or slows down the system clock’s drift rate so that (1) the system’s time is always correct and (2) there won’t be any subsequent time jumps after the initial correction.

Unlike NTP, SNTP usually uses just one time server to calculate the time, then “jumps” the system time to the calculated time. It can, however, have back-up time servers in case one is not available. During each interval, it determines whether the time is off enough to make a correction and if it is, applies the correction.

Clear as Mud?

If this is not completely clear, consider an analogy of comparing and adjusting a wristwatch to a clock on the wall. The wristwatch is analogous to the “client” device (like a PC) and the clock on the wall is the time server. With SNTP, you always look at the clock at pre-determined intervals. Let’s say one per hour. (As an aside, the act at comparing time for computer synchronization is known as a “poll.”)

When you think it is 12:00:00 you look at (poll) the clock to see that it is 11:59:57. You are three seconds fast, so you set your watch back three seconds. You do not do anything else until 1:00:00. You look again at the clock to see that it is 12:59:57 – again, three seconds fast — and again you set your watch back three seconds. Every hour, you reset your watch 3 seconds to be in sync with the clock on the wall.

From an error perspective, you are most accurate immediately after the poll and you progressively get worse. The maximum error happens immediately before the poll, when a sudden adjustment occurs, such as when time goes from 12:59:57 to 12:59:58 to 12:59.59 to 1:00:00 to 12:59:57.

If a maximum error of three seconds and the discontinuity of the time scale bothers you, consider the NTP case. Here, you want to react knowing that your watch is gaining three seconds every hour, so you don’t have to change it so often.

Simply compensate for the drift by using your error vs. time measurements. You do not need to use the same measurement period all the time. All you need to know is the rate and direction of the change.

After you have a pretty good feel for the drift, you can program your watch to adjust in real time. You want to make very small adjustments, so that at any given time you are in sync with the clock on the wall, without even looking at it.

Of course, the drift rate may change over time, so you do want to continually poll the clock, and apply the best correction you can come up with. And with that you get a wristwatch that is seemingly never out of synchronization!

Which One Do You Need?

It all depends on your application, but in general, SNTP clients should only be used where time synchronization is not critical for your systems. For all other clients, and for systems that will also serve time to other systems, you should utilize full NTP implementations to include reference selection and clock steering algorithms to maintain accuracy through the full timing path.

Looking at the time servers themselves, the selection of a time server that uses SNTP or NTP to serve time only should focus on whether that time server would ever synchronize to NTP as a primary or secondary reference — in which case, only full NTP should be used. To simplify things, SNTP should be used only at the start or end of the network timing path, and only at the end of the network timing path where time synchronization is not critical for your systems.

Mitigating an NTP Distributed Denial of Service (DDoS) Attack

By Pritam Kandel, Applications Engineer

Network time service is not something many businesses think about as a key component of their critical infrastructures. In fact, it is often overlooked entirely, and in error. As a result, the network architect or engineer often defaults to an easy alternative: using a server or network switch as the source of the network clock and synchronizing these sources to Internet time servers using Network Time Protocol (NTP). This white paper discusses the risks of, and alternative solutions to, “NTP Over the Internet.”


About Pritam Kandel

Pritam Kandel is an Applications Engineer with over a decade of experience working in design, assessment and implementation of TCP/IP routing and switching infrastructure for network cores/backbones, datacenters, Internet edge and WAN. He is experienced with maintaining IT infrastructure, including Internet peering and ISP services, MPLS and carrier networks, and VoIP global infrastructure. He holds certifications in CCNP, CCNA, JNCIA, MPLS Deployment, Alcatel Lucent and NIX platforms. Pritam is a graduate of the Rochester Institute of Technology with an MBA in Technology Management and holds a Bachelor of Engineering in IT from Pokhara University.

NTP Over Anycast. The Easy Way to Sync Clients and Servers

What Is NTP Over Anycast?

NTP (Network Time Protocol) over Anycast mode is a software technology that allows two (or more) NTP servers to sync clients via a single IP address.

NTP is a packet network-based synchronization protocol to sync a client clock to a network master clock.

Anycast is a networking methodology using standard routing protocols where messages are routed to one of a group of potential receivers via a single Anycast address, thus significantly simplifying the configuration management for the larger pool of clients.

NTP over Anycast, available in both SecureSync® and NetClock®, is a combination of the two concepts, allowing them to:

  • Associate one of their network ports to an Anycast IP address
  • Remove themselves as an available time source if the reference is lost or degraded, or vice versa

Though NTP clients typically need to be individually configured with the IP address(es) they are to sync with – even when using NTP over Anycast mode – this mode allows the clients to be configured with one address instead of multiple and lets the “nearest” available time server to respond to the request.

How It Works

  • Configure an Anycast IP address to any Orolia network interface.
  • NTP server responds to client requests as directed by the configured routing protocol so long as the time server is in sync.
  • NTP server becomes “unavailable” if the time server goes out of sync or a problem with its reference is detected. Client requests are directed to the “next-nearest” server, which is also configured with the NTP over Anycast address.
  • NTP server is automatically made available when synchronized.

When to Use NTP Over Anycast

Configuring NTP clients for mission-critical timing using NTP servers with static IP addresses can be problematic for large deployments across several network and geographic boundaries. Referencing an NTP server by hostname with existing DNS infrastructure can help, but still requires a lot of configuration.

For Anycast-enabled networks, the ability to route NTP requests to several potential servers via a single IP address, without any specific client configuration, offers the simplest, most reliable, most scalable approach.

Benefits to You

  • Reduce latency, increase availability, improve scalability of NTP deployments.
  • Simplify the management of a reliable wide-area NTP deployment with redundant stratum-1 servers.
  • Leverage the ability of a “smart” NTP server for a simple NTP client implementation.

How to Get NTP Over Anycast

 Contact Us for more information or, for currently fielded units, to receive the application software upgrade.

About the Author Sadie Nedo

Sadie Nedo is a global account manager at Orolia, where she supports the public safety market. For nearly a decade, she has specialized in helping PSAPs develop and deploy solutions that simplify the integration of precision timing and frequency into their critical infrastructures. She holds a bachelor’s degree in advertising and public relations from Rochester Institute of Technology.