Results 1 to 16 of 16

Thread: Concurrent FMMs, WAV and 1LINK failures

  1. #1
    Frequent Visitor
    Join Date
    Apr 2017
    Location
    Dubai
    Posts
    71

    Concurrent FMMs, WAV and 1LINK failures

    Total Neptune meltdown this morning. System went into a hard shutdown and on reset both FMMs, 1LINK and WAVs are failed. Modules are showing as failed in the module list.

    I am trying to figure out what happened and what I can do to recover.

    Both FMMs are blinking orange and blue, 1LINK is solid orange.

    I leave on a business trip for a week in 3 hours and am far from amused. Stripping out this £&@#% kit so I can revert to manual pump and skimmer control and have tankbuddy monitor for this week.

    This kit is simply not mission critical. I will have to rethink my control approach and take FMM and 1Link out of all feedback and control loops (actually take Apex out of ALL control loops and relegate to monitoring). And for sure WAV cannot be the only powerheads.




    Sent from my iPad using Tapatalk

  2. #2
    Frequent Visitor
    Join Date
    Apr 2017
    Location
    Dubai
    Posts
    71
    Update: 1LINK partially restored and WAV functioning. 1LINK ACC ports failed ON. Physically removed the FMM module with the flowmeters connected to it. FMM module with the optical sensors and floor leak detector still connected and seemingly communicating with Apex, but FMM status light blinking blue, green and orange. DOS function restored. Apex control via 10V output of Maxspects and over VarioS restored (after deleting all flow related logic). When I return from this trip will strip out Apex from primary control loops and use it as a monitor. If this failure happened three hours later it would have been down for at least a day. Back up battery on Maxspects will also now be upgraded.


    Sent from my iPhone using Tapatalk

  3. #3
    Frequent Contributor zombie's Avatar
    Join Date
    Dec 2013
    Location
    Denver, CO
    Posts
    13,176
    Contact support. This doesn't sound like something the forum can help you fix.

    You might be an engineer if...You have no life and can prove it mathematically.

  4. #4
    Frequent Visitor
    Join Date
    Apr 2017
    Location
    Dubai
    Posts
    71

    Concurrent FMMs, WAV and 1LINK failures

    Ticket is open. Will have to wait until I am home. Tankbuddy will check in this week. Seems a common failure mode that affected especially Apex. Not router, nor computers, nor TVs (amendment: the LaCrosse temp probes which are my temp alert backup disconnected at the same time and are still disconnected). Lucky I was literally in front of the tank working from home when this happened.


    Sent from my iPad using Tapatalk

  5. #5
    Frequent Visitor
    Join Date
    Apr 2017
    Location
    Dubai
    Posts
    71
    So can an EB6 outlet fail "on" even if Apex says an outlet is of and the programming code is "fallback off"? Yup. See pics. Heater outlet says off but clearly power is being supplied to the eHeim heater which is cycling on and off in a rough range of nearly .4C (under Apex it used to be within 0.1C).


    Sent from my iPhone using Tapatalk

  6. #6
    Frequent Contributor zombie's Avatar
    Join Date
    Dec 2013
    Location
    Denver, CO
    Posts
    13,176
    Quote Originally Posted by Charles0dxb View Post
    So can an EB6 outlet fail "on" even if Apex says an outlet is of and the programming code is "fallback off"? Yup. See pics. Heater outlet says off but clearly power is being supplied to the eHeim heater which is cycling on and off in a rough range of nearly .4C (under Apex it used to be within 0.1C).


    Sent from my iPhone using Tapatalk
    They can fail in the on position. It doesn't happen often, but that is why I suggest to everyone to have an internal thermostat or heater controller set 2 degrees above apex setpoint as a backup.

    You might be an engineer if...You have no life and can prove it mathematically.

  7. #7
    Frequent Visitor
    Join Date
    Apr 2017
    Location
    Dubai
    Posts
    71

    Concurrent FMMs, WAV and 1LINK failures

    Yup. Another lesson learned!

    Here's another; FMM module with optical sensors fails in such a way that all sensors are reported Open. So for now the Top off tank low level alarm is not working (this tank is full), display high alarm is not working and sump high alarm is not working.

    BoB came through this event fine. DDR and DOS are ok!

    Not quite the lesson I wanted to learn but basically BoB with float switches is more reliable than FMM with optical switches. Will keep that in mind when I revise my control, monitoring and backup design.

    Also impressed that Apex 2016 is itself seemingly limping along, although its own system health reporting is wildly optimistic. Its reporting all modules are ok when I have incontrovertible evidence that modules are either failed completely (Both FMMs; the one with the flowmeters is physically removed and deleted from the system) or are also limping along with failed outlets (EB6 heater outlet is failed On; 1LINK ACC 24V outlets are failed On).




    Sent from my iPad using Tapatalk

  8. #8
    NSI Member Krazie4Acans's Avatar
    Join Date
    Nov 2013
    Location
    US, Mountain Time
    Posts
    1,254
    The (OK) in the modules list is not a health indicator. It is an indicator of whether the module firmware is up to date or not. It will change to old if the Apex detects that the module has down level firmware that needs to be updated.

    If I were you I would save all of your programming into notepad or word or something like that and then physically remove all modules, remove them from the modules list as well and then add them back onto the Apex one at a time. Update the firmware for each module (yes even though they are reporting OK) and Verify that the module is working and that the programming is correct for that module before connecting the next module.

    I have a feeling that it might clear up the issues with none working modules and unpredictable switch states.

  9. #9
    Frequent Visitor
    Join Date
    Apr 2017
    Location
    Dubai
    Posts
    71

    Concurrent FMMs, WAV and 1LINK failures

    Thanks. Before I do that I am going to install a UPS with AVR to condition power and reduce EMI from mains power as I suspect that may be a factor in my highrise apartment with many many inductive loads in proximity on appliances, chilled water circulation, etc.


    Sent from my iPhone using Tapatalk

  10. #10
    NSI Member Krazie4Acans's Avatar
    Join Date
    Nov 2013
    Location
    US, Mountain Time
    Posts
    1,254
    Conditioned power is always a good thing although a pricey one. Do you suspect EMI surge or low voltage inductive load cycle? EMI surge is easily handled with a quality surge suppressed power strip. Low voltage clearly needs something more capable of conditioning the line voltage.

  11. #11
    Frequent Visitor
    Join Date
    Apr 2017
    Location
    Dubai
    Posts
    71
    I honestly don't know what gremlin I am chasing. Neptune suggested EMI and I don't have test equipment to verify or deny but I can't dispute the highrise I live in here in Dubai is EM rich. I suspect harmonics from inductive loads from my appliances, HVAC, neighbors appliances, building IT and chilled water services, nearby cell towers, RF leaks from cable, etc. etc. etc. Maybe solar flares, overflying aircraft radars and proximity to Iran (kidding). I have UPS's on all my freshwater and they have all been zero hassle.


    Sent from my iPhone using Tapatalk

  12. #12
    NSI Member Krazie4Acans's Avatar
    Join Date
    Nov 2013
    Location
    US, Mountain Time
    Posts
    1,254
    You're missing Lunar phase, Neighbors wifi hotspots, mobile vehicle hotspots, kids peeing in the buildings pool and tidal currents. Good catch on all the rest of them though. Oh and flushing toilets!

  13. #13
    Frequent Visitor
    Join Date
    Apr 2017
    Location
    Dubai
    Posts
    71
    Lol. All present and accounted for! Once I have had my shrink session with Neptune will update in about a week from now.


    Sent from my iPhone using Tapatalk

  14. #14
    Frequent Visitor
    Join Date
    Apr 2017
    Location
    Dubai
    Posts
    71

    Concurrent FMMs, WAV and 1LINK failures

    One interesting lesson I learned from my Apex 2016 system failure is that I had set it up initially to run on my wireless network with a backup LAN wired connection. That was a good thing because the wireless connection is indeed failed. Another lesson learned is I am running the Apex 2016 Controller just fine without the Powerbar. It's getting power from the 24V PS on DOS and the 24V PS on 1LINK. Today I happened across a 12V PS in my parts bin that fits the Apex 2016 perfectly. So now you know; Apex 2016 doesn't need a Powerbar for system power. I am inclined not to bother with the Powerbar at all and revert to only monitoring with the Apex.


    Sent from my iPhone using Tapatalk

  15. #15
    Frequent Visitor
    Join Date
    Apr 2017
    Location
    Dubai
    Posts
    71
    Update: After a long call with a very patient and helpful Neptune Technician the Powerbar EB6 is confirmed failed. My LFS will process the return and has provided a loaner EB6, which is working great.

    The Neptune Technician ran several diagnostic and repair programs which have restored 1Link and FMMs.

    For my part I added a UPS with AVR.

    I've reprogrammed Apex and removed all flow dependent statements. There is still some weirdness with FMMs which may require a follow up call.

    One footnote is now that I have the UPS monitoring and conditioning power I heard it beep over to its onboard battery in quick succession a couple of times last night. To drop mains and revert to onboard battery the mains voltage frequency or voltage must be substantially going out of limits during these transients. For every beep the UPS made I realize it was one less Apex reset. Lesson I guess is that even in (...or especially in!) a 60 floor residential highrise mains power is dirty, and may be dirtier than what the Apex is designed for. I won't run an Apex again without a UPS with AVR.


    Sent from my iPad using Tapatalk

  16. #16
    Frequent Contributor zombie's Avatar
    Join Date
    Dec 2013
    Location
    Denver, CO
    Posts
    13,176
    Power quality is usually not much of an issue in the states as our electric grid has the cleanest power on the planet, but I could see that causing problems in Dubai. The owner of my company does a lot of work out there and he has never had anything good to say about the power quality (probably a result of all the VFDs and power quality filters on the grid itself). Outside the US and Canada I wouldn't fully trust an apex without a UPS or power conditioning. Even in the states at least a surge suppressor is an absolute necessity to keep the apex safe from lightning strikes to the power grid.

    You might be an engineer if...You have no life and can prove it mathematically.

Similar Threads

  1. Replace WAV 1link cable?
    By rsumner in forum WAV Powerheads
    Replies: 1
    Last Post: 09-15-2021, 12:30
  2. eb832 and 1link WAV problem
    By Car2n in forum EnergyBars 832 / 632 / 8 / 6 / 4
    Replies: 5
    Last Post: 11-22-2018, 18:11
  3. WAV and 1Link questions
    By Muttley000 in forum WAV Powerheads
    Replies: 2
    Last Post: 03-24-2018, 08:19
  4. Question: DOS being powered via WAV 1link
    By vinnie in forum DŌS & DDR – Dosing and Fluid Metering System
    Replies: 2
    Last Post: 11-03-2016, 06:28
  5. Do I need a 1link for the WAV with the new Apex?
    By lou_raffael in forum A2 Apex/ApexEL and A3 Apex Pro/Apex/Apex Jr
    Replies: 3
    Last Post: 06-29-2016, 17:50

Bookmarks

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •