1. Hello everyone,

    For the past 48 hours, we have been experiencing network issues caused by one of our core systems and we have decided to temporarily limit the player count to avoid further complications.

    First, we want to be transparent and explain the issues we're dealing with;

    We have a program called MasterControl that handles pretty much everything like server balancing, parties, friends, server queuing, mystery boxes, transactions and much more.
    It's important to note that we haven't changed anything to that program for weeks, but for the past 2 days, we have seen MasterControl freeze randomly past over 50,000 players.
    We did a bunch of diagnostics and noticed a lot of critical items that need to be fixed. For example, yesterday we fixed the fact that Housing (of all things) was using up 20% of MasterControl's memory. However, we are still seeing issues today, so we are not done and we will be digging deeper in the next few hours.

    Historically, MasterControl issues are somewhat harmless, however, when MasterControl completely freezes, some vital parts of SkyBlock stops functioning - and collateral damage can happen such as people losing coins from their banks or auctions because the game needs to talk to MasterControl to verify the transactions. We are doing our best right now to gather the names of people affected, but it does not fix the core issue because MasterControl is often rebooted as part of some updates, so we need to spend time on a real solution and player cap in the meantime.

    Two main things need to happen;
    1- Fix MasterControl random freezes.
    2- Change the way SkyBlock handles transactions

    Right now, the team is currently rewriting a lot of backend systems on the server for SkyBlock and MasterControl. From the player's perspective - you! - mostly nothing will change, however, any annoyances coming from these issues will cease completely (hopefully!).

    That being said, code takes time. We have a lot to work on for this to be done, and we have a lot of profiling to do. For this reason, for the time being, we will be limiting player count on the network to give MasterControl some time to breathe. Increasing player count is always super exciting, but we prefer offering a quality experience first.

    I should mention that we released a dev blog today that is very related to our current issues: https://hypixel.net/threads/dev-blog-6-keeping-up-with-the-masses.2251554/
    Also, a post from Simon about SkyBlock specific issues:
    https://hypixel.net/threads/skyblock-issues-august.2258490/
     
    #1
    Last edited: Aug 19, 2019
    • Like x 257
    • Useful x 31
    • Hype Train x 12
    • Agree x 10
    • Dislike x 4
    • Creative x 3
    • Funny x 2
    • Helper Lapis x 2
    • Disagree x 1
    • Mod Emerald x 1
  2. Good stuff! Glad you are working on this as fast as possible!
     
    #2
    • Like Like x 31
    • Agree Agree x 9
    • Funny Funny x 1
  3. Alright, good luck to you and the rest of the dev team in pulling this off. Doesn't sound easy and I know I'd screw it up almost instantly so massive props to you guys. Also, I like how you guys are offering quality over quantity, it's very commendable.
     
    #3
    • Like Like x 17
    • Agree Agree x 11
    • Hype Train Hype Train x 2
    • Funny Funny x 1
  4. lopen

    lopen Well-Known Member

    lopen
    MVP+
    CowSquad COWSQD
    Officer
    Messages:
    13,016
    Thank you for the update on this, loving the fact you guys are communicating from behind the scenes to show you are working on making the show run smooth again
     
    #4
    • Agree Agree x 13
    • Like Like x 1
    • Funny Funny x 1
    • Creative Creative x 1
  5. good luck! thanks for the dev team to work on this as soon as most of them could.
     
    #5
    • Like Like x 6
    • Agree Agree x 2
    • Funny Funny x 1
  6. galactus

    galactus Well-Known Member

    Messages:
    5,382
    MasterControl sounds like a top secret FBI utility lol
     
    #6
    • Like Like x 14
    • Funny Funny x 8
    • Agree Agree x 3
    • Hype Train Hype Train x 1
  7. Kinda having the same issues as you guys with my server but on a way smaller scale! having an influx of players out of know where and not expecting it can cause havoc on multiple systems that were not designed for the massive influx of players. Keep up the great work!
     
    #7
    • Like Like x 3
    • Agree Agree x 1
  8. Good luck! :)
     
    #8
    • Like Like x 1
    • Agree Agree x 1
  9. Fascinating... At least now we can link this thread to any randoms who start shrieking about poor server quality :D
     
    #9
    • Like Like x 3
    • Funny Funny x 2
    • Agree Agree x 1
  10. I don’t understand any of this :D
     
    #10
    • Funny Funny x 4
    • Like Like x 1
  11. Mornin

    Mornin Member

    Messages:
    43
    Good luck...
    ;)
     
    #11
  12. Jacux

    Jacux Active Member

    Jacux
    MVP++
    Presto PRE
    Member
    Messages:
    278
    Thanks for the update! :)
     
    #12
    Last edited: Aug 19, 2019
  13. Wolf

    Wolf Well-Known Member

    Messages:
    10,150
    Hopefully games other then skyblock won’t crash randomly now.
     
    #13
  14. It's really nice having these update threads so we actually understand the problems.
     
    #14
    • Agree Agree x 2
  15. Good luck to the team to rewrite the code ;).
     
    #15
    Last edited: Aug 19, 2019
    • Agree Agree x 1
  16. Thanks for the clarification! :)
     
    #16
    • Agree Agree x 2
    • Like Like x 1
    • Creative Creative x 1
  17. Wishing the dev team good luck during this time.
    I'm sure all of this will be resolved soon enough!
     
    #17
    • Agree Agree x 1
  18. Hopefully this will put a stop to the confusion and complaining on the skyblock section.
     
    #18
    • Agree Agree x 1
  19. Thanks Hypixel! We all appreciate the work you put into the server we all love and enjoy.
     
    #19
    • Agree Agree x 1
  20. Nice.
     
    #20
    • Like Like x 1

Share This Page