Resolved
|
The issue was completely resolved on Feb 26th. Thanks for your patience.
Feb 28, 2025 5:50 pm
|
Monitoring
|
If you were impacted by this incident, please contact support by submitting a ticket from your account for an SLA credit. This would equal a 0.06% outage which would result in a credit equal to 10% of your monthly TrendSpider cost (because the outage was during market hours.) We will double this and provide a credit equal to 20% of your monthly service cost. If you were impacted, please contact support to request an SLA Credit under the TrendSpider Trader's Bill of Rights SLA.
Feb 26, 2025 5:15 pm
|
Monitoring
|
All services should now be restored. The team is monitoring to make sure everything stays online. A post mortem will be performed and we will publish our findings.
Feb 26, 2025 5:05 pm
|
Monitoring
|
We are now 90% back online. A handful of services must be manually checked before we can confirm. The team is working on that.
Feb 26, 2025 4:14 pm
|
Monitoring
|
The team continues to work to restore services. While they work, we wanted to provide a more detailed update about what happened. TrendSpider is designed in a modular way. Each aspect of the product is internally a microservice running separately. This means Charts and Alerts and Bots and Scanning and all other aspects of the system are separate components that work together. There was some type of networking issue in the data center that hosts the majority of our microservices which broke the microservices ability to communicate with each other. This caused some of them to appear disconnected or offline, even if they were running in the background. One of the affected service was the log-in and log-out service. As a result, users were unable to log in. We first attempted to restore internal connectivity, however when this failed to restore services, we moved to plan B which was to reload the entire platform on fresh infrastructure. This is a contingency plan that we have prepared for. Upon execution of this plan, service became accessible again. Now, user created data, such as drawings, alerts, saved scripts, etc. are being restored to the new clusters. We are very sorry about this and once we are back online we will conduct a full post-mortem analysis to understand what happened and to prevent similar instances from happening in the future. In the moment, while the platform is back online, some services continue to experience degraded performance. Our focus is on restoring them before doing any additional analysis on root cause. Thank you for your patience.
Feb 26, 2025 3:46 pm
|
Monitoring
|
Some users may continue to experience issues with saved content and other aspects, such as alerts. We continue to work towards full restoration. Thank you for your patience.
Feb 26, 2025 3:39 pm
|
Monitoring
|
The majority of microservices have been restored. Certain aspects of the platform are still being worked on, including but not limited to recalling of saved Visual Scripts and JavaScripts and other forms of user created content. All services will be restored soon fully. Thank you for your patience.
Feb 26, 2025 3:27 pm
|
Monitoring
|
System restoration continues...
Feb 26, 2025 3:19 pm
|
Monitoring
|
Most users should now be able to log back into the platform. Please note, some services may still not function. The system incident is not over, but it has moved from an outage to a degraded performance situation. We expect full restoration of services shortly. Please stay tuned to this page for updates.
Feb 26, 2025 3:18 pm
|
Monitoring
|
Services are starting to come back online now. Please note you may experience some errors with loading saved content in the short term. We are still working on full restoration.
Feb 26, 2025 3:10 pm
|
Investigating
|
We’re still focused on resolving the issue and ensuring service stability. At this time, we haven’t received new technical details, but we do know that some users can log in after refreshing their browser—even though a few services remain degraded. We appreciate your continued patience and will update you as soon as we have more information.
Feb 26, 2025 2:58 pm
|
Investigating
|
The team is still working to restore full service to the platform. Please bear with us.
Feb 26, 2025 2:39 pm
|
Investigating
|
The team is working to restore service to the platform. We apologize for the delay. This matter has proven complex.
Feb 26, 2025 2:26 pm
|
Investigating
|
Our team is still working on rolling out updates, and we have noticed that some users can log in after reloading their browser. However, please note that certain services are still experiencing degradation. We appreciate your continued patience and will provide another update as we have further news.
Feb 26, 2025 2:19 pm
|
Investigating
|
Our team is still actively working to resolve the networking issue affecting parts of the platform. We’ve made progress in diagnosing the root cause and are implementing fixes to restore service as quickly as possible. We’ll provide another update in about 10 minutes or sooner if we have significant news. Thank you for your continued patience.
Feb 26, 2025 2:05 pm
|
Investigating
|
The platform is currently still experiencing a networking issue that is disrupting service for some users. If you open the platform and wait, it may load. However, this is currently inconsistent. The team is all-hands-on-deck working to restore full service. Thank you for your patience.
Feb 26, 2025 1:46 pm
|
Investigating
|
The team continues to work to restore service.
Feb 26, 2025 1:39 pm
|
Identified
|
We are still working on the problem. That's our top priority.
Feb 26, 2025 12:54 pm
|
Identified
|
The platform is currently not available for some customers. We're working to remedy and restore service.
SUMMARY: There is a networking issue that is preventing some TrendSpider microservices from contacting others. This means for some users, the service will not load immediately. If you are trying to get in, please be patient. This issue is complex. The team is all hands on deck working to restore service.
Feb 26, 2025 11:20 am
|